• Please enter minutes on this page, with the newest at the top. Put a line between entries. Highlight action items in RED with a name/initials indicating who's responsible for the task and a due date, if there is one. For minutes prior to 2014, see the Archive of work through 2013.



Present: Linda, David, Ellen, Brendan, Dan, Sarah, Steve (minutes)

Notes from the look at the preliminary data analysis of the 2016 faculty user surveys (Brendan)

  • Teaching & funding by campus – compare to MINES data to explore relation between data sets

  • Open access: break out physicists, and how they are contributing to the overall numbers.

  • Importance/assessment of undergraduate information literacy skills Qs: 2010 – citations & ethical combined in one question? May need to explain for 2016 comparison. Can WR150, CGS, & Questrom faculty responses be pulled out for these?

  • Should be reviewed by the Library Task Force: Growing importance of info lit skills, but less satisfaction with skills of UG & grad students. Should the library be more involved in these activities?

Survey responders have been randomized and winners (5 UG, 4 G, 3 F) picked. Brendan will send the names to Linda, who will email w/ banner (running a draft by Ellen beforehand) asking them when they can come in to pick up their prizes. Pictures will be taken when possible & sent to Dan for posting & news item.

A look at how the data is being cleaned up (Brendan): Brendan has been meticulously recording everything he has been doing to clean up the data. This document is available on the Google Drive 2016 data folder as read-only

For our next meeting (5/26), complete Data Management Plan for (All):

     National library surveys including ARL, ACRL, IPEDS

    Triennial User Surveys



    Website Usage Patterns

    Visualized reports

    Published reports

    Data storage & sharing 

Next meeting: Noon to 1:30 on Thursday, 5/26, in the large Mugar conference room.



Present: Linda, David, Russ, Steve, Brendan (notes), Tom, Ellen, Dan, Konstantin

    • Announcements: Tom won two awards!  Congrats
    • Agenda:
      • Linda - Survey Results 
      • Dan - Publicity update
      • Brendan - Preliminary longitudinal analysis
      • Tom/Ellen/Brendan - Data Management Plan
    • Survey Results(Linda):
      • Response rate = 15.1% overall.  Surpassed Ithaka's
        • Lowest - Undergrad: 13.8%
        • Highest - Graduate: 16.6%
      • Problems with expired surveys in 3rd week. 16 k sessions "started" & expired. David saved the day! We resent the survey as a correction
      • Nicole requested we try not to use the Provost's name too much.  The fixed email went out with Bob's name attached.
    SK: Have to be careful to not use the Provost's name too much.

    SK: Expiration dates matter! Can't backdate on Qualtrics.

    Publicity Summary (Dan):
    • Collecting publicity locations & social media screenshots, storing them in a single location for future reference.
    • Social media may be working to reach out to people. There were bumps in the response rates on the days of retweets.
    • For future campaigns, we need non-targeted adverts and images for things like social media.
    • IRB doesn't seem to mind about the effect of social media skewing who's taking the survey.
    • Next time we should try to get onto email lists and student newsletter.

    SK: We should have a general image targeted for mass social media.

    SK: Medical campus is not good for postering.  There are digital displays though.

    Undergraduate results (Brendan):
    • All data captured up to 3/30.
    • Adjunct data was very low. Called "Part time contingent".
    • Changes made to libraries have definite effect on numbers.
    • Brendan will add highlights later
    • Less library instruction may have overarching effects on perception about the libraries.
    -Brendan: To pull out following data-
      • Frequency of online use of resources by school/other demographics.
      • Instruction by School

    Graduate results (Brendan):
    • Lots of Ph.Ds
    • Decrease in journals -> looking at archives and other resources now?
    • General trend of journals being less important, books being more important.
            -Brendan: To pull out following data-
      • Contributions by Schools
      • Resource importance by School
      • Nonimportance of Resources School
      • Review of new questions
      • Unsatisfaction with hours.
      • Unsatisfaction overall.

    TO DO:
    Tom - Send out the dmp information again again
    Brendan - to meet with Konstantin about comment analysis.
    Next Meeting: 4/20 at 12:45 at LAW. Come early for the NISO webinar on Library Assessment.



                                                                                                        BD, TH, DB, LP, KS, EF, RS, SS (telecom), DF

                                                                                                        Next meeting: 4/7 Thursday  12:00pm-1:30pm  


                                                                                                        The survey has successfully launched on 3/15/16 There were just 17 faculty emails that bounced

                                                                                                        Thus far: 4% for undergraduate 680  average duration 20 minutes.Graduate students 5%  1100   (the top responders are GRS and Medical school) Faculty 5%  250 

                                                                                                        There is a concern that we won’t be able to reach the same 30% we did last time. A positive development is that there are more comments than last time

                                                                                                        The launch was staggered to allow for the potential intervention. But everything went everything very smoothly.

                                                                                                        Steve suggested that we should develop a second reminder which politely reminds patrons that they haven’t yet taken the survey.

                                                                                                        The Task Force - we had a discussion how to approach the communication strategies with the staff in regards to the work of the task force. Linda will contact Bob Hudson.


                                                                                                        Every school is covered – digital and physical posters. Reminders will go 3/22 and 3/29

                                                                                                        Posters are on the buses

                                                                                                        Dan is tracking all the social media. Five postings on Facebook and four on the Blog, and nine tweets.

                                                                                                        News announcements on some of the library web sites.

                                                                                                         Idea – can we use mascots (BU Red) in the next survey to boost our response rates.


                                                                                                        DATA MANAGEMENT

                                                                                                        Google Drive

                                                                                                        One of the folders is labeled “Data Management”

                                                                                                        Tom suggested two approaches – a number of meeting devoted to this topic or to work together on Google Drive and then have just a few meetings. 

                                                                                                        We are waiting for the response from Bob Hudson regarding the survey that we would like to put out to the library staff about data management

                                                                                                        Data sets and the issue of de-identifying the data - can we release de-identified our data. We can certainly release the survey instrument and reports.

                                                                                                        TO DO:

                                                                                                        ·       Review the Google Doc – do everything up to the policy (the first three questions)

                                                                                                        ·       Review Shirley’s comments



                                                                                                        BD, TH, DB, LP, KS, EF, RS, SS, DF

                                                                                                        • Minutes, announcements, changes to the agenda
                                                                                                          • Dan is presenting a poster on MINES at the 2016 Library Assessment Conference. If you want to go, talk to Dan
                                                                                                          • Brendan and Tom went to the Software Carpentry workshop, focused on using terminal and associated resources to analyze data. Upcoming workshop on Data Carpentry: http://www.datacarpentry.org if anyone is interested 
                                                                                                        • User Surveys
                                                                                                          • Contact in Provost’s Office
                                                                                                            • no contact as of yet, Bob has been in touch this week and we should have one soon
                                                                                                          • Pilot of
                                                                                                            • Mary McCabe, exempt application coordinator at IRB says we can go ahead and pilot after we submit the application. 
                                                                                                            • Question: Can we put in the communications from the Provost’s office without IRB having reviewed them, and then change in real time as needed? 
                                                                                                            • Qualtrics distribution convenience sampling testing still scheduled for 2/1
                                                                                                          • Demographics—will not use data from registrar b/c can get from provost 
                                                                                                          • Publicity
                                                                                                            • Images are fine and do not need to be changed.
                                                                                                            • We can do one poster each week on Warren towers and will try to put a lot up on the busses. 
                                                                                                            • Dan will get in touch with each school and find out who controls digital display boards in lobby
                                                                                                            • Training
                                                                                                            • Everyone needs to complete the financial conflict of interest (check your email) by 1/29
                                                                                                        • Changes to surveys
                                                                                                          • Change 2015 to 2016
                                                                                                          • change Marlene Alderman to Ronald E. Wheeler, wheelerr@bu.edu
                                                                                                          • iPad mini3 TO iPad mini4
                                                                                                          • BU Libraries Search should NOT be all capitalized because of Law search (generic name, not proper name)
                                                                                                          • Library Usage and Resources—ask “during current year” grad and undergrad are same, except undergrad omits “accessing scanners”—> no reason for this difference, so add to undergrad?
                                                                                                          • Task force will not be a surprise. Provost’s Office will add it to the invitation, so we do not need to put it in the methodology 
                                                                                                          • Brendan to make changes & corrections to surveys
                                                                                                          • Brendan will make a read-only archive of all our things for the sake of preservation
                                                                                                          • Ellen to read all four survey instruments for typos after Brendan does overlay 
                                                                                                        • Other
                                                                                                          • We will be having Zepheira using an ALMA feed to transform our catalog records into linked data. It will have geotagging and google analytics so we can track things, to make the data in our records discoverable through Google. We should plan to survey before and after. 
                                                                                                          • Will this boost circulation and usage data? 
                                                                                                          • Mike and Jack will host a conversation on this initiative soon—please attend!

                                                                                                        Minutes - Assessment Committee – 12/11/2015

                                                                                                        Next meeting: Thursday, January 7 at 12pm -  Mugar Admin Conference Room
                                                                                                        No Announcements
                                                                                                        Present: Dan (minutes), Tom H, Tom C, Sarah, Konstantin, Ellen, Linda (chair), Brendan, David, Steve

                                                                                                        TechQual results at BU, and a coming National Survey of Student Engagement - Tom C

                                                                                                        • The library related question on TechQual is question number 17.  The results were pretty typical, but the idea is to score better next time.  This is a gap measurement tool (minimal vs perceived vs desired level of service).
                                                                                                        • Tracey of IS&T plans to run this again every two years.
                                                                                                        • There were 6 pages of comments collected, and Tom C will send them to the AC via email.
                                                                                                        • Ellen will get a copy and also we decided the DAG should receive them too.
                                                                                                        • National Survey of Student Engagement, this is past the planning phase at BU, but Tom has been invited to revisit the library related questions on it by Denise Mooney, as it will likely be given again in the future.

                                                                                                        Applying a Data Management Plan to MINES report material - Tom H, Ellen, Brendan

                                                                                                        • package up data from MINES survey for public use
                                                                                                        • idea would be to facilitate further insights for library or other colleagues
                                                                                                        • should include a readme.txt file that describes data formats and methods
                                                                                                        • Jupyter was mentioned as a tool that helps to capture workflow
                                                                                                        • Action item: Brendan will go through MINES data material to be made public.
                                                                                                        • Tom will move forward with this, and look to bring an example back to the committee

                                                                                                        Staff survey on storing and sharing data - Tom H, Ellen, Brendan

                                                                                                        • Idea is to survey staff about their data storing and sharing activity
                                                                                                        • LP- who will answer this survey and why?  All staff, for all work output?
                                                                                                        • TH - outcomes could come in form of AC recommendations to Bob H.
                                                                                                        • could ask questions such as: what is your essential function?, are you a supervisor or individual contributor? do you produce training or tutorial material?  do you have departmental guidance on where to store data output? does the data you collect end up going into a national survey (arl stats)?
                                                                                                        • Results will start conversations about best practices, especially in the situation that someone leaves the job and needs to pass on data processes to other folks
                                                                                                        • should the survey be anonymous or not?
                                                                                                        • Tom will move forward with this, consider methodology and bring thoughts back to the committee

                                                                                                        Item for next agenda: Linda

                                                                                                        • More opportunities to present, publish, and share the MINES Report and data REPORT

                                                                                                        Minutes - Assessment Ctte – 10/30/2015

                                                                                                        Present: Tom, Konstantin, Ellen, Linda (chair), Brendan, David, Steve (minutes)

                                                                                                        Logistic of 11/19 all-staff meeting:

                                                                                                        • Location: Law 525
                                                                                                        • Bob Hudson will welcome, with possible introductory comments by Marlene or Russ Sweet
                                                                                                        • Joanne Pierce, Learning Assessment, Provost Office will be invited by Linda
                                                                                                        • Refreshments will be arranged by Cathy M.
                                                                                                        • Ellen will arrange room & technology
                                                                                                        • Bob will ask that new staff be introduced (by supervisor)
                                                                                                        • Action Item: Assessment ctte members should plan to arrive at 1:30, spread out and welcome people; we should encourage our staff and colleagues to attend

                                                                                                        MINES REPORT

                                                                                                        • Acknowledgments: Will use titles rather than affiliation; please give David your preferred title
                                                                                                        • Action item: Send your title that you would like used in the acknowledgment section.
                                                                                                        • LP will provide affiliations for Terry Plum, Brinley Franklin, and Martha Kyrillidou, and for the ITS folks to acknowledgments
                                                                                                        • David will work up an executive summary
                                                                                                        • Change ‘to March’ to ‘through March 3rd’ or ‘through February’
                                                                                                        • David will work on the library visits portion of the “Where” conclusions
                                                                                                        • Tighten up ‘What is an online section,” second paragraph
                                                                                                        • Consistent use of ‘chart’ and ‘figure’ (i.e., there may be multiple figures within a single chart)
                                                                                                        • Delete the sentence intro descriptions for “Why” (top of p. 5) and “Where” (p. 6) and “When”; or move some of the description of these up from the conclusions
                                                                                                        • Strengthen 2nd paragraph under “Who” recommendation           
                                                                                                        • Action item: Give the report a final read through, make sure everybody is thanked/acknowledged.


                                                                                                        Minutes - Assessment Committee - 10/22/15

                                                                                                        Attendees: DF, RS, LP (chair), EF (minutes), BD, DB, TH, SS


                                                                                                        ·         Harvard assessment librarians will be visiting Monday, November 2 at 11 AM

                                                                                                        ·         Tentatively scheduling all-staff meeting to present MINES report November 19th from 2-3, but not advertising yet

                                                                                                        Action items: Linda, Brendan, and Tom will read the appendix of the MINES report and bring feedback. David will rewrite the conclusions section based on today’s feedback, in a cohesive and coherent style, and send around a new draft for feedback

                                                                                                        Next Meeting: 10/30 in Law Library Conference Room (third floor) 2-3:30PM. 

                                                                                                        MINES Conclusions Discussion--general

                                                                                                        ·         Primarily discussion of compiled comments available here

                                                                                                        ·         Will remove “use of library resources for course work is closely tied to the academic  calendar, while use for research is more constant” because it doesn’t really get us anything 

                                                                                                        ·         KS had suggested quotes from Franklin & Plum article. Decided not to use actual quote, but incorporate ideas in these and Tom Casserly’s quote on library use, and note that it is in line with what other schools have found (undergraduates are using library extensively, but they are not accessing online resources from within the library)

                                                                                                        ·         Also need to change “the libraries will” to “the libraries shall” because it’s a recommendation, not a mandate or promise

                                                                                                        ·         All of TH comments are included in LP comments, as are EF’s so majority of discussion focused on LP’s suggestions. 


                                                                                                        ·         We will incorporate Dan’s feedback on use of the term “online sessions” re grad students to avoid inferring overall use

                                                                                                        ·         Mines data does not tell us how heavily resources are being used. This is an indicator of library use, similar to gate counts, but there is limited ability to extrapolate overall use from these user sessions—need to reword title “number of online sessions initiated in year” in figure

                                                                                                        ·         We can tell who is using things, but not comfortable saying “heavy” or “significant” use because who can tell if it really is?

                                                                                                        ·         Change emphasis on use to who, rather than what—all members of community are using resources in what we would consider significant ways. Overall, all users are using print and electronic collections—no one group has been left behind by shift from print to e. 

                                                                                                        ·         Is there a way to include ARL data so that we can demonstrate how we rank in reference to our peers? (i.e., to determine how heavy our usage is) This would be difficult, because the data would need to be normalized (FTE and volume count)

                                                                                                        ·         “From the number of online session, we roughly infer usage”—> then include that all groups are making use of the resources (as opposed to roughly estimating amounts of usage by group), which also gets us away from qualitative statements about amount of use. 

                                                                                                        ·         Collections are part of our services, we should also discuss services.  

                                                                                                        ·         May also need to examine services to graduate students

                                                                                                        ·         “online journal collections” should be changed to “online collections.” Also need to include “online services” in this.

                                                                                                        ·         Reword to something like “library’s preferred mode of acquisitions is serving all patron groups equally well/satisfactory” b/c presently, implications don’t actually deal with WHO.

                                                                                                        ·         Also add that because grad students are heaviest user group, should consider/examine their services and needs to make sure we are adequately meeting them. 

                                                                                                        ·         TH: Who should have two conclusions and implications: one discussing everyone, and then also a second conclusion and implication on graduate students. 

                                                                                                        ·         Suggested text: Currently, collection building is largely focused on faculty, which serves undergrad demand (through course reserves). Unlike UT and UW, we don’t have separate undergrad and grad libraries, so question of what are we doing to satisfy graduate needs isn’t currently getting answered. Worth investigating, although ultimately may not be a big deal. 


                                                                                                        ·         RS: suggests focusing energies on getting greater integration b/w library resources and primo and course management systems

                                                                                                        ·         Need to rewrite conclusion to be clearer that it includes both types of students (grad and undergrad).

                                                                                                        ·         Also need to include that faculty secondary use is preparing for teaching.

                                                                                                        ·         Graduate student teaching use number (1%) makes sense, when you consider what and who is teaching—law, med, business don’t teach, usually teaching canned courses so not necessarily research for teaching. 

                                                                                                        ·         Break into three conclusions and implications: WHY coursework, WHY teaching, and WHY research (w/in research, tease out funded vs unfunded research). 

                                                                                                        ·         undergrad coursework implication: information literacy instruction. DF to pull from survey to determine whether or not they’re having trouble, how we can better support their efforts 

                                                                                                        BD did analysis of how WHY varies by location—tasks groups are likely to be doing in library vs other locations. Implications for services—need to make them available at points of access. Services based on research and scholarship should take this into account, especially for undergrads, since they’re not doing research when they come to the library, and by the time they are doing research, they’re no longer coming to the library. Determined to be interesting, but not worth putting into report. Save for internal use. 


                                                                                                        ·         MINES data and survey data conflict re value of space à Illustrates disconnect between library space use and library collection use.

                                                                                                        ·         If not using library space for access to collections, what are they using it for, and how well are we doing it? à External survey data brought in to help answer this.

                                                                                                        ·         MINES shows one side of this (use of collections)—> not saying library space is not important, just saying it’s not being used for what we thought it was being used for

                                                                                                        ·         Add to table on library resource use: library gate count to help illustrate that just because resources are not being used in the library, does not mean the library is not getting used. Once gate count is added, we can bring in survey data on importance of library. 

                                                                                                        ·         Suggest adding note on non-homogeneity of populations between libraries, because some use of e-resources comes from within library. 

                                                                                                        ·         Suggest adding to conclusions that just because students use online resources, they are still looking for human services—reference help, etc. 

                                                                                                        ·         Implication: need physical version of MINES to issue short survey to every 100th person coming in to the library. 

                                                                                                        ·         Need to include comments and conclusions on use of physical space to include professional students/schools

                                                                                                        Minutes - Assessment Committee -  October 15, 2015 

                                                                                                        Attendees: Linda (chair), Brendan (minutes), David, Russ, Virtual Konstantin, Dan, Tom, Steve, and Sarah

                                                                                                        Notifications and Changes to Agenda - 
                                                                                                        • University Assessment Workshop happening October 16th, 9am-12pm.  At least Linda & David attending.
                                                                                                        • Harvard assessment librarians planning a visit to discuss creating a library-wide assessment plan, they would like our input
                                                                                                        MINES Publicity Timeline
                                                                                                        • Goal is to send to Bob ASAP.  Bob will send them to Provost and to staff.  Upload to website shortly after that.
                                                                                                        • Plan for a staff-wide meeting around Thursday November 19th, perhaps 2-3:30. Will cover findings, methodology, conclusions and next steps at least.
                                                                                                        • Staff-wide meeting planning to happen a week or two before presentation date.
                                                                                                        • David agreed to do main presentation; Linda volunteers to voluntell us to present if more presenters are necessary.

                                                                                                        Discussion and Review of Final Draft – Conclusion and Appendix

                                                                                                        • Tom C.’s thoughts:
                                                                                                          •  Move the importance of MINES paragraph to the top of the introduction since it is most important.
                                                                                                          • "A lot of online use by undergraduates is in library, and this is a significant figure to quote for 'library-as-space' purposes"
                                                                                                        • General:
                                                                                                          • Our conclusions should each lead to implications. We can split up conclusions, for example, by the W’s of the report, or by user group.
                                                                                                          • Be wary of drawing transient or dubious conclusions.  Does the data really lead to implication?  Are statements about user groups reflective of reality (e.g. off “campus” use for faculty in MED campus)?
                                                                                                          • It’s okay to mention our results are surprising. It adds weight to implications and our analysis.
                                                                                                        • Discussion regarding library resources vs collections.  David will change resources to collections where appropriate and 
                                                                                                          • “Collections are a major resource of libraries” maybe in the conclusion?
                                                                                                        • Minimize clunkiness of first and second conclusions if we’re keeping 
                                                                                                        • Considering whether to have an executive summary.
                                                                                                        • Maybe pull in data from previous surveys/statistics.  Ex: “In the 2012 survey, this was the satisfaction of the grads with our online journals. Coupled with the MINES data…”
                                                                                                        •  Libraries aren’t *only* used for our resources.  Accessing our website is almost exclusively for our resources.  We shouldn’t make too many location extrapolations.
                                                                                                        • Expand on what we would change for our next MINES run-through.
                                                                                                        • Conclusion/implication suggestions:
                                                                                                          • Grad students are the heaviest users -> We should focus acquisition efforts on graduate students? Is that possible?
                                                                                                          • “Repurpose space for undergraduate use in library” – check with Bob first.
                                                                                                          • We should bring in how the data points to an increasing demand for online services, hence our changes to staffing.
                                                                                                          • Maybe a statement about improving resource accessibility, or help finding and interpreting our online resources. 
                                                                                                          • Current purpose conclusion : ‘Graduates use resources most for coursework, and then research -> We should expand (in-class ?) information literacy in context of the courses, or we need collaboration efforts with teaching faculty to ensure they have available and needed tools, or provide/supply support for research services such as data.
                                                                                                        • Fill out suggestions for conclusions and implications by the next meeting on the Google Doc: MINES Report - Conclusion Suggestions

                                                                                                        NEXT MEETING: 10/22/2015 at 2:00, location TBD

                                                                                                        Minutes - Assessment Committee -  October 8, 2015

                                                                                                        Attendees: Linda, Russell, Konstantin, Dan, David, Brendan, Tom, Steve, Ellen, and  Sarah

                                                                                                        Minutes: Tom

                                                                                                        1. Minutes, changes to the agenda, announcements - None
                                                                                                        2. MINES Report Discussion - 
                                                                                                          1. General Feedback - Everyone thought the new draft looked very polished, it was well done, and there were a lot of positive remarks. The following items were small edits to the draft:
                                                                                                            1. Add difference between sessions and use - make it clear in the introduction that sessions are not equivalent to use. Example of gate counts could be helpful to tell about use .
                                                                                                            2. Lack of BMC category - We will look at adding this to our next survey.
                                                                                                            3. Add full-text library downloads to introduction page - remove library web page views 
                                                                                                            4. Add gate count to the "where" section or introduction from ARL stats - potentially with language about library as space
                                                                                                            5. Look at the 6 paragraph in the introduction - maybe pulling that paragraph up to start the report 
                                                                                                            6. Perhaps review the conclusions to highlight 
                                                                                                            7. Inconsistent use of gradients in graphs 
                                                                                                            8. Check BU Libraries Assessment Committee in report title and throughout the document
                                                                                                            9. Review the chart colors and their look in black and white 
                                                                                                            10. Add paragraph about what online resources we are talking about in the introduction and remove Top Online Destinations
                                                                                                          2. Conclusions - we will look at the conclusions of the MINES report at our next meeting.

                                                                                                        Our next meeting is: Thursday, October 15 from 2:00-3:30 in the Pardee Conference Room

                                                                                                        Assessment Committee Minutes - September 4, 2015

                                                                                                        Attendees: Linda, Russ, Dan, David, Brendan, Tom, Steve, Ellen, Konstantin

                                                                                                        Minutes: Tom

                                                                                                        Reporting for the ARL annual statistics has begun. The deadline is Oct. 1 for ARL and about a week prior for BU. Linda will be coordinating these efforts and will be contacting people appropriately.

                                                                                                        MINES Survey Report:
                                                                                                        We had a great conversation about the David's initial draft for our MINES Survey Report. The group all agreed the initial report was an excellent start. We spent the remainder of the meeting discussing the document's structure, language, and overall purpose.

                                                                                                        Some general thoughts:
                                                                                                        - The group would like to conduct more analysis in research funding section of the document. This analysis might include a breakdown of research funding and library usage by audience and school.
                                                                                                        - We discussed that we need to be mindful that there is not a 1:1 relationship between online sessions and use. Additionally, we should clarify how the MINES data supports and supplements the data we already gather about usage of library resources.
                                                                                                        - We looked at restructuring the sections as: who (use), what (top online), when (use over time), where (location), why (purpose)
                                                                                                        - We determined that any charts information about the print collection should be symmetrical with the MINES data (same audience breakdown, same scale, and legend)

                                                                                                        Potential "high-level" recommendations:
                                                                                                        - Our conversion to "e first" is not leaving any group behind and that we should continue building services around our electronic holdings
                                                                                                        - Library services should assume that patrons are going to access our holdings online and outside of the library's walls
                                                                                                        - Coursework drives undergraduate and graduate use of the library and we should prioritize outreach efforts appropriately
                                                                                                        - We should continue prioritizing staff efforts and distribution around our electronic holdings
                                                                                                        Our next meeting is: Thursday, September 17 from 12:00-1:30 in the large Mugar Conference Room

                                                                                                        Assessment Committee Minutes 7/30/15
                                                                                                        Attendees - Linda (chair), Brendan (minutes), Dan, David, Ellen, Steve, Tom

                                                                                                        Next meeting: Thursday, August 6 from 1:00-2:00 pm (Report Review
                                                                                                        NEXT-next meeting: Friday, September 4 at 2pm

                                                                                                        To do-
                                                                                                        Linda to add Tom to assess-l,
                                                                                                                   to forward him the contact for research assessment at the Provost's office,
                                                                                                               & to research whether an evaluation of (inter)national library surveys has been done.
                                                                                                        Brendan to get circulation statistics for Lib Use Report.
                                                                                                        Dan to get webhit numbers for David
                                                                                                        David to contact Mike Ward about Primo searches.
                                                                                                        Ellen to forward link to Assessment Cookbook notification.
                                                                                                        Everyone to continue working on their initiatives.

                                                                                                        Assessing social media impact (Dan)

                                                                                                        Plan includes a scholarly lit review, student feedback, env scans of peer institutions, and approaches to mgmt.
                                                                                                        Include a collection of example posts.
                                                                                                        Finish gathering by November this year.
                                                                                                        Interviewing to finish March next year. (contact Mary McGowan, Ellen to share law contacts)
                                                                                                        Complete report by next year summer..
                                                                                                        Output: explanation of best practices and local policy recommendations.
                                                                                                                Dan mentions BrowZine, a tool for journal browsing.  Expensive but may be useful for faculty.
                                                                                                        MINES/Library Collection Use Report (David)
                                                                                                                Next meeting will be devoted to group review of the draft.
                                                                                                                Dan, Brendan, and Steve to assist with collecting statistics for the report.
                                                                                                                Looking to finish the report by second or so week of September.
                                                                                                        Data Security (Tom, Ellen, Brendan)
                                                                                                        we need an index of who's collecting what, and contact data.
                                                                                                                output: How libraries organize our data.
                                                                                                        Linda suggests putting our data in our institutional repository.

                                                                                                        Data Visualization (Tom, Brendan, Konstantin*)
                                                                                                                Efforts made to visualize SEL headcount data. Preliminary graphs will be made more interactive.
                                                                                                        Usage Stats (Steve)
                                                                                                        We need to set standards for how to put data into Alma.
                                                                                                        Collab between Steve, Ellen, maybe Jennifer and Tim.
                                                                                                        Will start working over the fall.
                                                                                                        Lots of stuff (Ellen)
                                                                                                        ACRL sent out request for "Assessment cookbook".  Due date is 8/6/15.  Ellen to forward link to all of us.
                                                                                                        Idea: putting together a professional reading collection.
                                                                                                        Steve - why not make a lib guide for them?
                                                                                                        Cornell U Library research and assessment unit - Check it out.
                                                                                                        InCites (Linda)
                                                                                                        Provost decided not to renew InCites.  They like Academic Analytics (strictly faculty).
                                                                                                        Library may push to get access to Academic Analytics.  We may be able to work with the company to get lib-interesting data.
                                                                                                        New Idea - Look at other national/international surveys (only a few) and institutions sharing the same assessment genealogy. Establish reasons 
                                                                                                                    for why we have assessed the way we have.
                                                                                                        Linda talked with Jillian "director of learning assessment for BU". Expect more communication in the future.
                                                                                                        Timeline, realistically start after mid October. output a report like MINES and recommendations.
                                                                                                        David - This should be published if it hasn't been done before.
                                                                                                        Data Storytelling Workshop (Tom & Brendan)- 
                                                                                                        Designed for journalism fields interested in data. It's a new field meaning we're not behind the trend.
                                                                                                        Showed off techniques for getting leads on stories, searching for datasets, cleaning data with tools.
                                                                                                        Most important lesson: keep focus on people. Add stories to the data to engage.
                                                                                                        Libraries stand to inform people about data storytelling ourselves, workshops, guides (timed with seminar papers maybe).
                                                                                                        Journalism students may be a useful asset for us.


                                                                                                        Assessment Committee Minutes 5/29/15
                                                                                                        Attendees - Linda (chair), Dan (minutes), Brendan, David, Tom H, Steve

                                                                                                        Next meeting: Thursday, June 18 at 2pm

                                                                                                        Announcements: none

                                                                                                        1.  Data management (Brendan & Tom)

                                                                                                        Brendan and Tom presented a slideshow about the Data Management initiative. They examined at data classifications, possible data roles and overall goals at the Libraries.  They are planning to construct data policies that encourage folks to work with the Libraries on data issues.  Data permissions, training from IS&T for OneDrive, and centralizing library data were also discussed.

                                                                                                        2.  InCites (Linda & Steve) 

                                                                                                        Linda and Steve lead the discussion of this tool.

                                                                                                        Initiative: Exploration of InCites as an assessment tool
                                                                                                        A report about InCites addressed to the Library’s Assessment Committee will be written this summer. It will include a purpose statement, a description of InCites, an estimate of the current usage, and the potential of InCites for the Assessment Committee. The report will include recommendations for the Assessment Committee and the Library’s outreach efforts.
                                                                                                        Target Due Date:
                                                                                                        Preliminary Outline

                                                                                                        • Purpose
                                                                                                        • Description of Resource
                                                                                                        • Target Audiences
                                                                                                        • Limitations
                                                                                                        • Current Usage at BU
                                                                                                        • Environmental Scan of Use by Other Universities/Libraries
                                                                                                           - Literature review
                                                                                                           - Questions posted to assessment listservs, with follow up and summary about use (ARL, BLC)
                                                                                                        • Recommendations

                                                                                                        3.  Assessing Social Media Impact (Dan)

                                                                                                        This project will be a preliminary look into social media use best practices for libraries, and will result in a document that includes:

                                                                                                        • a scholarly literature review,
                                                                                                        • examples and feedback from students,
                                                                                                        • an environmental scan,
                                                                                                        • and approaches at various BU Libraries.

                                                                                                        "Being active in the social media sphere is important because it provides innovative ways for us to connect with users we may never see face to face on a personal and meaningful level." Fichter, Darlene, and Jeff Wisniewski. "How To Measure the Results of Your Activity on Social Media Sites." Marketing Library Services 23.2 (2009). 


                                                                                                        Assessment Committee Minutes 5/5/15
                                                                                                        Attendees - Ellen, Dan, Konstantin, Brendan, David, Tom, Steve, Linda (chair)

                                                                                                        Next meeting: Friday, May 29 at 1pm



                                                                                                        1.       Minutes, announcements, changes to the agenda

                                                                                                        2.       FY16 Initiatives

                                                                                                        1.        Organize and describe our data sources. (Ellen, Brendan, Tom)
                                                                                                        2.        MINES reports (David) 
                                                                                                        3.            Data visualization (Brendan & Tom)
                                                                                                        4.        InCites (Linda & eventually Steve)  
                                                                                                        5.        Impact of social media (Dan) 
                                                                                                        6.        Ongoing tracking  of usage statistics for e-resources- U-Stats (Steve)
                                                                                                         Next meeting  

                                                                                                         The Assessment Committee welcomed a new committee member, Tom Hohenstein.

                                                                                                         The following discussion has taken place on goals, timeline and scope FY16 initiatives. We've discussed data usage (f.ex. CRC vs. Medical, Law databases, etc.), data services (reference instructions, floor plans, hours, etc.), sources for data (Google forms, LibGuides, IPADS survey)

                                                                                                         Updates on ezProxy, Web of Science, Academic Analytics (not yet available to BU Libraries).

                                                                                                         Other topics we brainstormed:  comparison of our data to outside sets (f.ex. rationalization and normalization of our data in the context of outside data); collection of data (f.ex. focus groups - see also Primo Advisory Group, which right now focuses on usability testing); audience and appropriate products; resources and tools at BU (Hariri Institute for Computing and Computational Science and Engineering, Digital Initiatives, etc.)

                                                                                                        In the next 15 minutes the group brainstormed ideas for the FY16 Initiatives (see list above)

                                                                                                        1.       Organize and describe our data sources. (Ellen, Brendan, Tom)

                                                                                                        2.       MINES Reports (David)

                                                                                                        3.       Data visualization (Brendan & Tom)

                                                                                                        4.       InCites (Linda & eventually Steve)

                                                                                                        5.       Imapct of Social Media (Dan)

                                                                                                        6.       Counter complianbt denials

                                                                                                        7.       Ongoing tracking  fo Usage Statisitics for e-resources- UStats (Steve)

                                                                                                        8.       Environmental Scan of Data being collected by BU Libraries

                                                                                                        Reminder: Anita Green Scholars - starts sooner this year

                                                                                                        Assessment Committee Minutes 4/17/15
                                                                                                        Attendees—BD, TH, SS, JR, JA, RS, EF, DF, KS (via skype), MW, LP, TL

                                                                                                        Next meeting: Monday, May 4 at 2PM. Focus: substantive conversation on what type of data we want to use.


                                                                                                        1. Brendan to report re EZProxy Logs meeting with Scott Macomber
                                                                                                        2. Linda re Eddie from Sourcing on use of Tableau
                                                                                                        3. Discussion re use of priorities for next yea


                                                                                                        Brendan re Scott Macomber & EZ-Proxy Logs

                                                                                                        • EZProxy records two types of files— .log (records everything) and .spu (records first place they land after login)
                                                                                                        • Mines captures URL data approximately every 100th person.
                                                                                                        • Not totally clear what EZproxy captures, Scott is working with OCLC to sort this out. 
                                                                                                        • Brendan has 2 files—log for Jan 1 has 86,000 logs. BD discussed cleanup process for .log files—there is a lot to be done, so we would need to limit amount we asked for/looked at
                                                                                                        • .log files show username, so needs to be removed. .log captures all webpages opened/any motion by user id (i.e., when they click the back button)
                                                                                                        • .spu captures first session through EZProxy i.e., if you went to multiple BU resources in one session, it only captures the first one
                                                                                                        • Brendan will compare host files to MINES and see if there is anything we can pull. Also going to tie to Google Analytics data to assess search path based on timestamp to educate us on how to improve our search capabilities. 

                                                                                                        Should we really do this? Is it worth it? What questions are we trying to answer?

                                                                                                        • BD recommends one year, max b/c of time involved.
                                                                                                        • JA recommends running a call with Python to pull user group before stripping username, then TH recommends scripting to automate
                                                                                                        • BD does not recommend options listed on OCLC website due to age and ease of use, so we are likely looking at scripting something by hand
                                                                                                        • Side note: JA is not sure how long we will have EZProxy, b/c depends on vendor requirements
                                                                                                        • Can’t get user group info from elsewhere, but what do we get out of that? 
                                                                                                        • SS is not sure how he would use resource info for collection development.
                                                                                                        • Ustats does already give us stats on what is downloaded. 
                                                                                                        • How are people finding our resources? Google Scholar? Primo? 
                                                                                                        • User behavior is interesting, but could this be gathered through snapshots of log, rather than analysis of all the data 
                                                                                                        • RS: we don’t have COUNTER-compliant data, so this would help us some representation of what is being used for renewal decisions; would also help with ARL questions re usage stats. LP—we need COUNTER compliant data. RS—OCLC Parse holds itself out as COUNTER-compliant. JR—reports themselves are not COUNTER-compliant, they just use COUNTER recommendations/format

                                                                                                        BD, DB, and LP met with Eddie from Sourcing re Tableau

                                                                                                        • He is willing to be a reference person for our use of Tableau
                                                                                                        • We don’t know the other products for data visualization, so not recommended we make any decisions just yet
                                                                                                        • Looking at this for presenting information in meetings with staff, university administration, etc.
                                                                                                        • It is very expensive, although it can be setup to grab data from multiple sources, but we may not neneed this functionality 
                                                                                                        • Meeting at 2PM Tuesday on Tableau, can be viewed in Mugar Admin Conference Room if interested 
                                                                                                        • BD to ask Rachel Lewellen of UMass Amherst if she will share some of her MINES/Tableau data 
                                                                                                        • We will hold off on purchasing Tableau for now because it has a steep learning curve, and we don't currently have an exact need for its functionality 

                                                                                                        Key focus for next year: data and getting it out to users


                                                                                                        Assessment Meeting Minutes 
                                                                                                        Ellen, Dan, Konstantin, Brendan, David, Sarah (minutes), Steve, Linda (chair)

                                                                                                        Next meeting 4/17 (Friday) 2-3:30pm. Linda will send out room info.

                                                                                                        Eddie from Sourcing is using Tableau, coming to meet with some of group 4/7/15 2pm small conf rm

                                                                                                        Reviewed what was covered in last mtg

                                                                                                        ARL SPEC kits – Konstantin
                                                                                                        ~4/year on hot topics are accepted. Written by ARL member libraries generally. All ARL libraries are asked to fill in survey, then data compiled and report written by authoring institution. Access to 2003-current will require a subscription. Earlier years are FT in Hathi trust
                                                                                                        2015 topics include one on scholarly output assessment activities
                                                                                                        Steve notes that in the past the examples/best practices included in SPEC reports were one of the more useful parts. So just the TOCs can be useful to know who might have examples (that may be available for free from their sites).
                                                                                                        May be worthwhile to see what assessment ones have been done.

                                                                                                        David – Analytics
                                                                                                        Look at patron searches in BULS, can this inform reference and instruction?
                                                                                                        Mike Ward says google analytics of page are currently better than Primo/ex Libris built in ones. No examples or models exist that DF found to assess the queries (how to do it, or if it’s useful). DAGS may be using some small parts of this for improving searches/results/displays
                                                                                                        DF doesn’t think it’s a high priority now, might be interesting to look again in a year when ex Libris analytics may be better.

                                                                                                        Brendan – MINES
                                                                                                        “We downloaded all the data from MINES, and there’s a lot of it” - over 7K responses. Completion rate around 91% last time it was calculated. BD cleaned up some bad data (probably statistically insignificant anyway). Added some databases hit based on connection URL for Ebsco. Discussed having BD create a couple other graphs. DF, BD, and LP will work on how to create reports from this data. DF- audiences: brief report for outside/above library, more detailed for internal use or sharing with other libraries. Potentially present to grad research council. Adding in data from outside survey? (# of potential users per school, etc). need context for reader in reports. What would we do differently if we ran MINES again? Longitudinal snapshot?
                                                                                                        Should BD work to draw out more detailed specific databases from aggregators? – consensus was no b/c of potential quality of data issues.

                                                                                                        Dan –Tableau Trial
                                                                                                        Uploaded some spreadsheets (ebrary stats Steve had sent out) and then explored some graphical display options. Lots of online support. Dan did look at a few example videos, but mostly was just exploring. Showed us 3 visualizations he’d worked on. Definite interest in learning more about it.

                                                                                                        Assessment Meeting Minutes 03/09/2015      
                                                                                                        Present: Linda (head), Brendan (minutes), Dan, David, Ellen, Sarah, Konstantin, Russ, Steve

                                                                                                        Next Meeting: Monday 03/23/15, Mugar Conference Rm.

                                                                                                        Action Items:

                                                                                                        • Linda: Ensure student permissions for archived publicity files.
                                                                                                        • Sarah: Send out fact sheet files and publicity contact list.
                                                                                                        • Brendan: Get EZProxy logs from Scott Macomber and fiddle with the data.
                                                                                                        • Everyone: Think more about the future of the committee.

                                                                                                        • Nothing new from Provost -> 2015 Library Survey will be postponed until 2016.  The surveys and how to implement them are being saved to a read-only dropbox archive for future reference.
                                                                                                        • Brendan showed the structure and contents of the archive.  Showed the contents of the survey manual.

                                                                                                        Individual Research Topics
                                                                                                        • Tableau and Data Warehouses [Dan]
                                                                                                          • Tableau is more visualization-driven than other data analyzers.
                                                                                                          • Rachel Lewellen of UMass Amherst does a lot with Tableau.  Uses it for presentations.
                                                                                                          • It's potentially another tool people will need to learn to use and manage among all the other tools people should learn to use, but don't.  A data warehouse may be wasted effort, and moreover money.
                                                                                                          • People don't want unprocessed data.  Give them facts and graphs that are useful by themselves.
                                                                                                          • Individualized dashboards would be the most effective, but they're labor intensive.
                                                                                                          • The Anita Greene Scholars are great graphic designers, but we need to schedule time with them since they're in high demand.
                                                                                                          • BU Libraries don't have a single go-to person for statistics and data (like Rachel Lewellen).  We should reconsider this.
                                                                                                        • Fact Sheets [Sarah]
                                                                                                          • Some fact sheets tell a story instead of numbers.  This is a good model for public fact sheets
                                                                                                          • We can compare library rankings and university rankings to highlight library contributions to BU's success.
                                                                                                          • Consider tying our delivered stats to the University strategic plan.
                                                                                                          • What data we share and how we share it depends on the audience.  A statistic filled fact sheet for sharing with libraries and provost; applied department-oriented figures for colleagues; fun and informative infographics for the users.
                                                                                                          • Maybe a newsletter would be useful for internal audiences.
                                                                                                        • EZProxy Usage Stats [Russ]
                                                                                                          • Usage is logged, but not analyzed.  "Does it log each link, or only each login?"
                                                                                                          • Program "EZParse" could be useful for analysis.
                                                                                                          • We may be able to get the total logs from Scott Macomber, or capture logs from a set of library computers.
                                                                                                          • Usable as a check against the MINES data.
                                                                                                          • Some concern that our EZProxy format will be difficult to analyze by database or journal.
                                                                                                          • Brendan will look into the logs by next meeting.
                                                                                                        • U-Stats and Cost per Use Data [Ellen & Steve]
                                                                                                          • Cost per use difficult to measure; depends on how we define use.
                                                                                                          • What data should we be collecting and how should we store it?
                                                                                                          • Consider creating a "best practices" procedure for data.
                                                                                                          • We should look at how other schools handle data.
                                                                                                          • There's a problem porting data between Ustats and Alma.  Alma Analytics may be the intermediary.
                                                                                                        • OUT OF TIME!  Next week: Konstantin with the ARL Spec Kit, David with "Analytics and Us", and a MINES analysis update from Linda and Brendan.  Next meeting: 3/23 in the Mugar conference room.

                                                                                                        Assessment Meeting Minutes 02/23/2015     
                                                                                                        Present: Linda (head), Dan (minutes), Brendan, David, Ellen, Sarah, Konstantin, Russ
                                                                                                        Next Meeting:
                                                                                                        Monday 03/09/15, Admin Conf Rm.


                                                                                                        • The opt-out open access policy passed Univ. Council.
                                                                                                        • Russ is leaving the committee; Ellen will be continuing to represent Law.
                                                                                                        Action Items:
                                                                                                        1. All - monitor for 2015 Survey developments as appropriate.
                                                                                                        2. All - be prepared at the next meeting to discuss future directions as assigned below.

                                                                                                        Update on Survey 2015

                                                                                                        We are starting to run low on time to accomplish the 2015 survey this Spring.  If we are not able to accomplish it, our new target would become Spring 2016.  As this is a distinct possibility, all of our materials are now well archived (thanks Brendan) in Dropbox and other locations.  There is also a "readme" file that explains what documents and materials can be found in the archive, and describes their purpose.

                                                                                                        Discussion of future work
                                                                                                        At the university level, there is a renewed focus on assessment systems for the evaluation of learning outcomes, and the library will be participating in this in the near future.  In light of this Committee’s charge and goals and the Libraries’ strategic plans, what other assessment activities would improve library services and resources.
                                                                                                        • [Ellen] - U-Stats.... cost per use data....  (discuss with Steve, and Jennifer from Law)
                                                                                                        • [Konstantin] - group of specialists to respond to the ARL Spec Survey > ARL Spec Kit
                                                                                                        • [Dan] - centralized point of access for all things library assessment (data warehouse?  Tableau? see * below)
                                                                                                        • [Sarah] - creation of Fact Sheet(s) for university constituents, such as bu.edu/research/  (needs examples, environmental scan)
                                                                                                        • [Linda] - MINES - now one year old at BU
                                                                                                        • [Russ] - investigate what ezproxy logs can tell us about use (and possibly save to U-Stats)
                                                                                                        • [David] - what can google analytics tell us about our library website users?  about our BU Libraries Search users?  about collection use (and possibly save these to U-Stats)?
                                                                                                        • Social Media outreach efforts - tracking twitter, facebook accounts
                                                                                                        • Know just what we do collect and should let decision-makers know vs. what should we be collecting and should let decision-makers know
                                                                                                        • leadership role for all library assessment activities
                                                                                                        • assessment tied to GPA trend
                                                                                                        • focus groups

                                                                                                        * From the ARL-ASSESS listserv:

                                                                                                        Data Management and Visualization Webcasts

                                                                                                        Data visualization brings library stories to life. Using Tableau software, libraries may better harness, analyze, and report their data to internal and external stakeholders. Join us for a three-part series on data management with Tableau, presented by experienced assessment librarians.

                                                                                                        Sarah Murphy, Ohio State University

                                                                                                        Tuesday, March 3, 2015, 2:00–2:30 p.m. EST

                                                                                                        Register online
                                                                                                        free of charge

                                                                                                        The Ohio State University (OSU) has used Tableau since 2012 to support a number of assessment projects. Examples include a series of interactive dashboards that query, analyze, and deliver transactional data to subject librarians to support their collection development and engagement activities. To deliver library data of interest to the broader academic community, the Libraries has also started to embed interactive, downloadable Tableau dashboards via select OSU Libraries websites.

                                                                                                        Sarah Anne Murphy is coordinator of assessment for The Ohio State University Libraries. She earned an MLS from Kent State University in 2000 and an MBA from The Ohio State University in 2008. She has published two books and several papers related to library assessment.

                                                                                                        Jeremy Buhler, University of British Columbia
                                                                                                        Tuesday, March 10, 2015, 2:00–2:30 p.m. EDT
                                                                                                        Register online
                                                                                                        free of charge

                                                                                                        The University of British Columbia (UBC) Library began using Tableau in 2013 to analyze longitudinal LibQUAL+ results. The resulting online display highlights changes in survey results over time and facilitates the comparison of responses across several user groups. Building on the success of this visualization, UBC Library is now using Tableau to encourage library staff to engage with, think about, and find new applications for library metrics.

                                                                                                        Jeremy Buhler, MLIS, is assessment librarian at the University of British Columbia Library, where he has worked since 2011.

                                                                                                         Rachel Lewellen, University of Massachusetts Amherst
                                                                                                        Tuesday, March 17, 2015, 2:00–2:30 p.m. EDT

                                                                                                        Register online
                                                                                                        free of charge

                                                                                                        Tableau helps the University of Massachusetts (UMass) Amherst Libraries staff to both analyze and understand data. Selectors may access purchasing and use data without having to maintain or manipulate spreadsheets. Using Tableau for the analysis of a consortial e-book project minimizes duplication of effort and provides a common perspective for understanding the data. Interactive visualizations expedite access to live data from MINES (Measuring the Impact of Networked Electronic Services), and make it easier for staff to use circulation, service desk, and gate-count data for operational purposes.

                                                                                                        Rachel Lewellen is assessment librarian at the UMass Amherst Libraries. She has been using Tableau since 2012

                                                                                                        Tuesday, April 21, 2015

                                                                                                        2:00–3:00 p.m. EDT

                                                                                                        Webcast—Data Management and Visualization with Tableau

                                                                                                        Discussion and Q&A with Sarah Murphy, Jeremy Buhler, and Rachel Lewellen


                                                                                                        Brendan, Dan, David (chair pro tem), Ellen, Russ, Steve (minutes)

                                                                                                        Next meeting: 2pm, 2/9, Mugar conference room

                                                                                                        To do:

                                                                                                        -Steve will serve as Sarah's backup to resolve the capitalization of Library, Libraries, library, libraries through the three surveys

                                                                                                        Bob Hudson updated the committee on the approval status, that the Provost has been given copies of the surveys to review.

                                                                                                        Survey pilots: Went well, with almost all 'piloteers' taking and completing the survey.

                                                                                                        The committee then reviewed comments from the pilot (see Brendan's 'Review of Encountered Pilot Errors and their Fixes' handout) and corrected errors as necessary.


                                                                                                        Dan, Russ, Brendan, Steve, Ellen, Konstantin (minutes), David, Linda (chair)

                                                                                                        Next meeting: 2pm, 1/26, Mugar conference room

                                                                                                        To Do: 

                                                                                                        - Konstantin and Ellen - identify 2 staff who are willing to participate in the pilot

                                                                                                        - collect and review responses from pilot testers

                                                                                                        -Sarah still needs to resolve the capitalization of Library, Libraries, library, libraries through the three surveys

                                                                                                        We’ve reviewed display option on the iPad.  It looks OK.

                                                                                                         Discussion of remaining issues:

                                                                                                        QF15, QG2a: Open Access -  we’ve agreed to the changes for points one and two regarding articles published

                                                                                                        Resolved the wording for library search – FQ5 -  copy from FQ5, GQ7 and UQ6 - 

                                                                                                        FQ6, GQ6, UQ6 -  our agreement: “Research assistance …”  “General assistance”

                                                                                                        FQ9 and QG13 - “archiving papers and other publications in digital archives” – include SSRN

                                                                                                        “Dear Faculty …” text is finalized

                                                                                                        We’ve finalized the email list for pilot participants.

                                                                                                        Include some staff to participate in the pilot – Linda will email the link


                                                                                                        Dan, Russ, Brendan, Steve, Ellen, Linda (chair), David (minutes)

                                                                                                        Next meeting: 2pm, 1/20, Mugar conference room

                                                                                                        To Do

                                                                                                        • Ellen: Check survey on iPad
                                                                                                        • Russ: Propose wording for questions about subject librarians/staff assistance/research assistance


                                                                                                        Linda will be on vacation Jan. 26th - Feb. 8th

                                                                                                        Should know Provost decision Jan 22nd.


                                                                                                        Looked at Linda’s proposed emails for survey recipients. Ellen had minor edit that was accepted and made by Brendan.

                                                                                                        For pilot, Brendan recommends that we we just use a non-individualized link for pilot. Can test panels separately amongst ourselves.

                                                                                                        Brendan looked at display logic affecting data; there should be no real problem.

                                                                                                        There was some editing of Linda’s emails for pilot participants, recorded by Brendan.

                                                                                                        Which surveys should staff see in the pilot? All of them.

                                                                                                        Linda brought up possible issue with viewing survey on iPad? David couldn’t reproduce on his iPad. Ellen will check it on hers.

                                                                                                        Went over various comments and suggestions for surveys. Brendan captured changes made.

                                                                                                        Russ will propose wording for Subject librarians/staff assistance/research assistance.

                                                                                                        Will use student workers for undergrad pilot.

                                                                                                        Next meeting is all about the pilot, and we plan to send it out after meeting.


                                                                                                        Linda (chair), Brendan (minutes), Steve, Dan, Sarah, Ellen, David , Konstantin, Russ

                                                                                                        Next mtg: 01/12/15 2:00-3:30, Lg. conf. rm.

                                                                                                        To do

                                                                                                        • Sarah: Ensure Libraries is capitalized/punctuated properly across surveys
                                                                                                        • David: Check for wording consistency across surveys.
                                                                                                        • Dan: Respond to Ken about our decision on splitting up teaching/research/clinical.
                                                                                                        • Linda: Draft up emails sent out with the pilot and the real survey.
                                                                                                        • Brendan: Review survey options, flow, and distribution plan.  Determine if display logic will mess up analysis.  Prepare pilot panels.  Modify surveys for pilot commenting.
                                                                                                        • EVERYONE!: Review ALL three surveys, and try to find 3 undergrads, grads, and faculty each (as appropriate), and email their info to Brendan.

                                                                                                        AGENDA: 1. Review previous meeting 2. Plan pilot


                                                                                                            BD finished changing the surveys.  Questions have been updated as discussed.

                                                                                                            Linda made chart of each survey's section and question flow.  It all looks good and makes sense.


                                                                                                            Linda likes convenient sampling.  “Try 3-5 each"

                                                                                                            We need people from both campuses.

                                                                                                            Steve mentions that data may be difficult to parse due to display logic.

                                                                                                                Brendan will check if this applies to us.

                                                                                                            Panels will be created for the reviewers

                                                                                                            For undergrads, don't use student workers, but their friends are okay.

                                                                                                            We should mention we’re collecting demos in the intro.

                                                                                                            Show Marlene, Mary, and Amy the survey before pilot.  “We thought you’d like to see this before it goes out.”

                                                                                                            Everyone’s Spring recess ends after March 15th. Target release date: April 2nd.

                                                                                                        Advertising Update

                                                                                                            Sarah showed off the adverts.  Very nice.  Still some tinkering here and there.  Lots of places we’re advertising, still waiting on even more locations.

                                                                                                            Russ: "Can we modify our websites for advertising the survey?"  Linda: “IRB says about 1 advert in each library is fine.  More than one will shift our user group         toward the patrons."

                                                                                                        Scheduling: Mondays 2-3:30 in the big conference room for the next 5 weeks, EXCEPT January 20th, 2-3:30 instead of January 19th. 

                                                                                                        Meeting adjourned.  


                                                                                                        Steve, Dan, Brendan, Sarah (minutes), Ellen, David (Chair), Konstantin

                                                                                                        next mtg: 1/5/15 2pm conf rm.


                                                                                                        Pilot Surveys

                                                                                                        OA Questions

                                                                                                        SDS presented new wording for OA questions on grad and fac (FQ36, GQ2a)

                                                                                                        Regarding articles you have published: [Respond Yes/No/Unsure]

                                                                                                          • Have you published an article in an open access journal?
                                                                                                          • Have you paid an additional fee to make an article open access?
                                                                                                          • Have you deposited an article in OpenBU (BU's institutional repository)?
                                                                                                          • Have you deposited an article in a discipline-based repository (e.g., PubMed Central, SSRN, arXiv)?

                                                                                                        On the faculty survey, the open access factor on the question "How important are the following factors in your decision on where to publish journal articles?" will read: 'Journal allows open access (immediately or after an embargo)'.

                                                                                                        DF suggested a small wording change: move “have you” after the “regarding articles you have published” and start each question with the verb (published, paid, deposited).

                                                                                                        Info lit Questions

                                                                                                        DB reports from conversation w/ Ken Liss

                                                                                                        already implemented wording on guides/tutorials

                                                                                                        separating teaching from clinical/research purpose (FQ 32,33,43,45) - KL interested, we were not open to making this large a change at this time

                                                                                                        consider in survey keepers- separating teaching from research from fac, or teaching from study for grads.

                                                                                                        Streaming Media

                                                                                                        GQ17 (could be added to fac FQ 45 as well) Streaming Media

                                                                                                        SDS - yes wants this question

                                                                                                        fac interested in JOVE; Krasker DVD based, is there interest in other?; Lots of audio, not much video currently

                                                                                                        BD has concern about “streaming media” wording could be interpreted different ways (synchonous vs asynchronous, live streaming vs netflix style)

                                                                                                        switch to “online video and audio resources”?

                                                                                                        “streaming audio and video” 

                                                                                                        decided on wording:

                                                                                                        “Access to streaming audio and video resources” for both GQ17 & FQ45

                                                                                                        Review of Questions/issues from BD’s email 12/15

                                                                                                        discussed, BD will make changes

                                                                                                        Assessment Meeting Minutes 12/15/14

                                                                                                        Attendees:  Linda (head), Dan, Brendan, David, Ellen, Steve, Sarah, Konstantin (minutes), Russ, Vika (called-in)

                                                                                                        Next Meeting: Monday, December 22, 2pm Admin. Conf. Rm.

                                                                                                        NEXT MEETING: Agenda

                                                                                                        1.            Question “streaming media” 
                                                                                                        2.        Report from Open Access group  (Sarah and Steve)
                                                                                                        3.         Discuss Ken’s second suggestion
                                                                                                        4.        Respond to Brendan’s email (all)

                                                                                                        12/15/14 Agenda

                                                                                                        1.       Minutes (thanks Ellen for the terrific minutes), announcements, changes to the agenda?

                                                                                                        2.       Discussion with Vika about our OA questions (Sarah)

                                                                                                        3.       Brendan’s overview presentation and outstanding questions (Brendan)

                                                                                                        4.       Discussion about Ken’s feedback on the survey’ Information Literacy questions (Dan)

                                                                                                        5.       Publicity files, Brendan’s suggestion about publicity

                                                                                                        6.       Do we want to add an augmented services question to the undergrad survey?

                                                                                                        7.       Pilot

                                                                                                        Brendan’s overview:

                                                                                                        Brendan walked us through all the major and minor changes. (see Brendan’s power point presentation). 

                                                                                                        The group discussed question “During that past academic year at BU have you …. 4th option: “Worked on a research project including data”.  We’ve decided to keep the wording as is.

                                                                                                        Double check if IS&T term is clear to the medical users -  include BU/BUMC IS&T ? …. (Konstantin)

                                                                                                        We’ve discussed the advantage of mentioning “Blackboard” in a question which asks about “course system”

                                                                                                        Sarah brought up the point that Ken’s suggestion that library guides need to be reincorporated in the undergrad and graduate surveys.

                                                                                                        We’ve discussed open issues concerning service augmentation.  Russ suggested to change the word “augmented” to “enhance”. We all agreed.

                                                                                                        We’ve decided to discuss the potential of removing “media streaming” during the next meeting.


                                                                                                        Discussion with Vika about OA questions:

                                                                                                        Sarah summarized Vika’s concerns and Vika clarified that we shouldn’t focus just on “open access journals” vs. open access in general.

                                                                                                        Vika suggested to formulate a separate question that asks about Gold and Green access.

                                                                                                        Discussion followed. Steve argued that we’re trying to find out if our patrons are making their work open access rather than publishing in the “open access” journal.

                                                                                                        We’ve decided to modify the question so that we capture “immediately or after embargo” wording.

                                                                                                        Question “Have you ever submitted an article to: ….” -  We’ve decided that we have to consider the wording in the context of Vika’s suggestion.

                                                                                                        Sarah emailed Vika’s suggested wording.

                                                                                                        Ken's suggestions

                                                                                                        Dan discussed Ken's suggestions, but we still need to revisit them.

                                                                                                        Augmented Services for the Undergraduates

                                                                                                        Russ suggested that we substitute the word "augmented" with "enhanced". The group unanimously agreed.


                                                                                                        Assessment Meeting Minutes 12/8/14

                                                                                                        Attendees: RS, ER (minutes), LP, BD, DB, SS, DF (for last half)

                                                                                                        Next Meeting: Monday, December 15, Admin. Conf. Rm.

                                                                                                        We focused on a group review of the completed graduate and faculty surveys

                                                                                                        -Grad & Faculty Surveys will NOT have a “demographics” bloc header—decision is that is it not necessary to prep users to answer personal questions.

                                                                                                        Sarah is setting up a phone in with Vika for next Monday’s meeting to discuss the Open Access wording on the survey and what data she would like us to collect.

                                                                                                        Things to cover at next meeting: Open Access questions and language with Vika, input from Ken on Information Literacy, conventions/consistency of language across all three surveys with Brendan, review the three surveys in their entirety, data management questions with David, and discuss the pilot


                                                                                                        On satisfaction and importance questions, IMPORTANCE rating will be asked first, and then SATISFACTION rating

                                                                                                        -Brendan is looking at alternative phrasing for “information literacy” and also looking at the three surveys together to see where language should consistent, but isn’t—he will present these at 12/15 meeting

                                                                                                        Faculty Survey

                                                                                                        -David is working on language for Research Data questions

                                                                                                        -“Research data” option has been added to question about “during the past academic year, have you…” and set up as display logic, so only survey takers who have worked with research data will see questions about storage and sharing

                                                                                                        -Question on bibliographic managers may be moved to different block, but this was not yet decided

                                                                                                        -Question on assigned readings and how students access—to be reviewed with Steve and Konstantin at 12/15 meeting

                                                                                                        -Dan is following up with Ken on the language for the information literacy question (FQ30)—this language must be to us by 12/15

                                                                                                        FQ32—discussed switching assessment and importance (to importance and assessment) but decided not too, since it would be leading, but are aware we may now be comparing apples and oranges

                                                                                                        -“Information literacy” is library speak, but may have to remain

                                                                                                        Grad Survey

                                                                                                        -We agreed to remove “other” from the importance & satisfaction re library resources questions

                                                                                                        -We agreed to add an option on current awareness to our question on augmenting library services—this question will also be copied to the faculty survey

                                                                                                        -We changed the question on library use and space (group and individual study space, access to computers) to focus on HOW graduate students use the library resourceà this change needs to be copied over to the undergraduate survey

                                                                                                        Assessment Meeting Minutes 11/17/2014     
                                                                                                        Present: Linda (head), Dan (minutes), Brendan, David, Ellen, Steve, Konstantin, Russ
                                                                                                        Next Meeting:
                                                                                                        Monday, 11/24, Admin Conf Rm.


                                                                                                        • Discussion of the Information Literacy questions with Ken Liss is delayed for one week.  Ken will join us next week and he and Sarah will lead that discussion.
                                                                                                        • The 2015 Undergrad Survey is tentatively complete.
                                                                                                        Action Items:
                                                                                                        1. Faculty Survey Subgroup should note the discussions in the high level review below and make necessary edits.
                                                                                                        2. Undergrad Survey Subgroup should be prepared to walk the group through a preview of that survey at the next meeting.

                                                                                                        High Level Review of Faculty Survey 
                                                                                                        • On Q32 (importance of resources) - The subgroup should remember to add the satisfaction scale to these (but not the use scale, as this is the faculty survey).
                                                                                                        • We decided to move the section on publishing & OA to near the end, but not past the last 'Overall' section.  Brendan accomplished this during the meeting. 
                                                                                                        • On Q43 (importance of services) - The subgroup decided to add Q44 (the satisfaction with same services) to this question, positioned horizontally next to it.  Not yet accomplished.
                                                                                                        • On Q45 (augmenting services) - We added a data management "statement" (note: this is the Qualtrics term for a row within a question).  The subgroup will work out the examples given and match the wording of the question on the grad survey.
                                                                                                        • For the Info Lit block of questions, the subgroup (actually, all subgroups) will monitor the results of our coming conversation(s) with Ken.  We did note it would be great to fold these 4 questions into 2.  Also noted the 'Providing correct citations' statement became 2 statements for the 2012 grad survey, and really could be 3 (providing appropriate citations, providing correctly formatted citations, and using information in an ethical manner).
                                                                                                        • On Q47 (overall contributions) - The scale needs to change to match the grad survey's scale.  We also added an "overall career success" statement.
                                                                                                        • At the end of the survey we considered other questions (Brendan noted these in Q51):
                                                                                                          • Put Bib manager question on all surveys (see undergrad).    
                                                                                                          • Take Faculty version of reserves question. "How do you direct students to resources you assign?"
                                                                                                          • Don't need first part of bib manager questions. Reevaluate options.


                                                                                                        Assessment Committee Meeting, Monday, November 10, 2014

                                                                                                        1.       Minutes, announcements, changes to the agenda

                                                                                                        a.       No changes were made to the minutes of the last full committee meeting submitted by Ellen

                                                                                                        b.      The following announcements:

                                                                                                                                                                       i.      Brendan noted a possible completion rate problem in the MINES survey, but he observed—and the rest of us agreed—that it did not represent a significant problem.  Even we had to accept the lower rate, it was still quite high and not worth intervention.

                                                                                                                                                                     ii.      Brendan also noted that in Qualtrics we can make notes and comments within the draft questions themselves, allowing us to keep all of the committee’s work product there and make it more convenient to read everything in one place, not only now but in the future.

                                                                                                                                                                    iii.      Linda announced that Ron Yeany and his team at IS&T have offered to look into whether they can help in extracting useful usage data from Exproxy.

                                                                                                        c.       No changes were suggested to the agenda proposed by Linda.


                                                                                                        2.       Continue discussion about usage, importance, and satisfaction and what changes we should make on our surveys.

                                                                                                        a.       Continued and reviewed the ongoing discussion we have had about whether to use usage or satisfaction in our survey. David and Steve quickly reprised the issues for us.

                                                                                                        b.      Konstantin registered his concerns about using importance and offered the white paper from the Customer Satisfaction Research Institute, along with four other relevant articles in the literature, as food for thought in our consideration when considering the use of importance in our survey.

                                                                                                        c.       Steve noted that if we moved away from importance we would lose the ability for longitudinal study, since we used importance in the first round of surveys.

                                                                                                        d.      We returned to the question if we could not use all three factors in the survey as suggested in the last meeting.  Russ queried if we could use the pilot period to test if the three factors could all be used in the survey without making the question too long and cumbersome, confusing the test taker.

                                                                                                        e.      Linda noted that we may have more room in the surveys for a few longer questions given that we are hoping to collect demographic data from the Registrar and Human Resources, rather than by asking the respondents.

                                                                                                        f.        Sarah observed that if we used all three factors that would give additional insight into patron feedback.

                                                                                                        g.       Consensus:   all three elements will be used in the pilot on questions related to services.


                                                                                                        3.       Demonstration/review of the Graduate Student Library Survey

                                                                                                        a.       Graduate Student subcommittee:  have had a high level discussion and are now having a more targeted discussion.  The subcommittee has met twice and needs additional time to discuss issues, but that can follow this discussion by the full committee.

                                                                                                        b.      A question was raised whether we can use the panels in Qualtrics for the distance education and online learners.

                                                                                                        c.       Question 8:  Brendan wondered if the wording is loaded; Linda did not believe it was; Steve suggested using “Please rate the importance” as a way to resolve the possible loaded nature of the question.  The question was also raised if the elements surveyed are really separate or do they represent a skillset (or many different skillsets).  Sarah suggested getting Ken Liss’s insights on this.   She also recommended inviting him to our next meeting to get his thoughts here as well as on other info literacy questions.

                                                                                                        d.      Question 9: on first line it was recommended to use just BU Library Search and then on the second line use “Databases” and search tools.  Russ and Konstantin also recommended using the examples of HeinOnline and PubMed, as was similarly done on the faculty survey.  We also considered whether we needed new examples of Social Media beyond Twitter and blogs, or need any at all.  The consensus was to keep the two existing examples.

                                                                                                        e.      Question 17:  Sarah wondered if we should not change the wording to be more like UW’s analog question #8, to be more user-centric.  We agreed that the subcommittee in its next meeting would investigate this further and make recommendations to the full committee.


                                                                                                        4.       Progress report on the Faculty Library Survey

                                                                                                        a.       We have had a high level committee discussion and one subcommittee meeting discussion.  Issues in the subcommittee, principally about the use of usage or importance, were then discussed by the full committee at the last meeting as well as in the early portion of this meeting.


                                                                                                        5.       High level review of the Faculty Library Survey

                                                                                                        a.       Additional high level discussions need to occur.


                                                                                                        6.       Next steps?

                                                                                                        a.        Undergrad survey is very close to being finished.  Needs an additional half hour of the full committee’s time.

                                                                                                        b.      Grad survey needs discussion in two more meetings.

                                                                                                        c.       Faculty needs two more meetings, although we may be able to shorten that time inasmuch as we are utilizing language developed in the new grad survey in corresponding sections in the faculty survey.

                                                                                                        d.      Should we include post-docs in our survey, and how?



                                                                                                        Assessment Meeting Minutes 11/3/14

                                                                                                        Present: BD, DF, LP, ER (minutes), SDS, DB, RS, KS

                                                                                                        Next Meeting(s):

                                                                                                        Graduate Subcommittee: Friday, November 7, 1-2:30, Mugar Small Admin Conf. Room

                                                                                                        Undergrad Subcommittee: Monday, November 10, 12:30-2, Administrative Offices, Mugar, small conference room

                                                                                                        Full Assessment Committee meeting, Monday, November 10, 2-3:30, Administrative Offices, Mugar, large conference room


                                                                                                        Brendan gave an update on the post-survey information we will use to capture entries for the gift cards, and he will send it around.

                                                                                                        There have been no changes to the undergraduate survey since our reserves discussion last week, and the undergraduate subcommittee will update us at the next meeting.

                                                                                                        Action Items:

                                                                                                        Whole committee: need to look at the larger information literacy question issues with Ken before we can decide on which word to use

                                                                                                        SRS, ER, KS: Add question about subject librarians to grad student survey

                                                                                                        All subgroups: make survey questions closer/more parallel across the board (see slide 11 where DF has compared the same questions across the three surveys, highlighting the different language we use)

                                                                                                        Undergrad survey/undergrad subcommittee: we have lumped together questions on physical things and services, and these should probably be two separate categories (see slide 15)

                                                                                                        Discussion re David’s presentation on usage vs satisfaction vs importance:

                                                                                                        Presentation (with DF’s recommendations on what word to use with which question) is on the wiki

                                                                                                        Overall: we are considering adding satisfaction questions to the faculty survey, but will ask undergrads only usage questions.

                                                                                                        Whether to continue distinguishing print vs ebooks? Key points:

                                                                                                        ·        Really two separate questions: (a) how important are these sources (print vs ebook) to you and (b) how satisfied are you with the selection we have at BU?

                                                                                                        ·        Do we care about responses if it will not change our format selection? (i.e., we are an e-preferred library)

                                                                                                        ·        Are we focusing on satisfaction with the format of our materials or our collection (e.g., “you don’t have what I want” vs “you have what I want, but not in my preferred format)

                                                                                                        o   Decided are trying to get more at satisfaction with the collection and so will combine the formats, which is what we did when asking about our journals collection

                                                                                                        o   May also add the word “collection” to the question to really get people to focus in on it

                                                                                                        ·        This question will also have a comments box

                                                                                                        ·        Determined there are no questions about e vs print materials we want to add, as we already know what we need to on that topic

                                                                                                        Importance vs Usage re: Library Services

                                                                                                        ·        Satisfaction questions are good for giving us longitudinal data, but are not particularly useful for capturing information from just one survey

                                                                                                        ·        How to capture usage?

                                                                                                        o   If faculty member is on sabbatical and we ask about the past year—will they say they have not used anything, when ordinarily, they heavily rely on our resources?

                                                                                                        §  Alternatively, will faculty pay attention to “current year” or look at their normal course of business?

                                                                                                        §  But “current year” should definitely be kept for ugrad survey, as it helps us to look at what resources each class uses

                                                                                                        §  Grant cycle is 3 years—if faculty are not writing grants in this current academic year, will usage questions underreport on resource usage/importance?

                                                                                                        o   Use of importance AND satisfaction questions together could give us a better idea of where we need to improve (e.g., less heavily-used resource may still be important, especially with varying workloads over the course of a faculty member’s career. Should we not limit to the current year for faculty, as it may provide an inaccurate snapshot?)

                                                                                                        o   What can this data really tell us? How would we tell if something is low use because they are dissatisfied or if it is low use because it is not an important resource?

                                                                                                        ·        Faculty use in past year vs importance to faculty—

                                                                                                        o   How to capture resources that are important but may not have been used in the past year and/or are not necessarily used on an annual basis?

                                                                                                        o   Issue: probably don’t want to ask faculty all three questions (is it important, do you use it/how often do you use it, are you satisfied with it?)

                                                                                                        ·        Issues with deriving usage stats from satisfaction data—e.g., on the undergrad survey, they reported satisfaction higher than their reported usage, but can also be important—the discrepancy itself provides useful information

                                                                                                        ·        “use is use”—usage questions wouldn’t tell us the type or quality of use (i.e., getting help with a paper vs asking us where the bathroom is)

                                                                                                        ·        Should we ask all three questions, but cut down the number of services we’re asking about to five or so?

                                                                                                        ·        Services for which usage is declining may still be important (i.e., ILL)

                                                                                                        ·        Currently, the usage question is a yes/no, not a likert scale. What is the value of asking it this way?

                                                                                                        o   If we give up the usage question, we give up being able to report usage

                                                                                                        o   Likert scale is also misleading, because what might be frequent for one user wouldn’t be frequent for another

                                                                                                        ·        Tip people off at the beginning with a header like “for the following services, we will ask you to think about whether you use these services, how important they are to you, and how satisfied you are with them” – makes people more likely to answer questions, rather than getting bogged down by never ending questions and quitting the survey?

                                                                                                        ·        Can usage be pulled from other areas so we don’t have to ask the question (i.e., get ILL stats from ILL librarians)?

                                                                                                        o   This question helps capture the perception of services they think they use

                                                                                                        o   Capturing usage in the survey helps us correlate use and importance in a way that cannot be captured with other data sources on use

                                                                                                        ·        Degree of use—do we care? à kind of an open question

                                                                                                        ·        Importance might also measure unique, non-repetitive events, as opposed to regular use of an itemà potential for skewed/misleading results

                                                                                                        Overall: committee members agree on main points with the exception of the use vs importance questions

                                                                                                        Suggestions for resolving this issue:

                                                                                                        ·        Make our ideal survey, then edit

                                                                                                        o   Need to be careful we’re not making an entirely new survey and losing comparability to prior surveys

                                                                                                        ·        Create a matrix of Use, Satisfaction, and Importance with two categories: Facilities and Services

                                                                                                        ·        Is there a more elegant way to ask about what library you use (that would help eliminate the need for separate facilities questions, so we only need to ask about services)?

                                                                                                        Can we bring in some topical questions (i.e., data management) that we can change each time?                    




                                                                                                        Assessment Meeting Minutes 10/26/14
                                                                                                        Present: SDS, DF, KS, ER, LP, BD, SRS (minutes), RS, DB 


                                                                                                        Brendan went to Google Analytics workshop on GA for libraries in Washington DC, and reported back on some of what he learned:
                                                                                                        We should make analytics user friendly for our  librarians to access/use
                                                                                                        See if IT planning on upgrading to newest version
                                                                                                        Suggested consolidating “properties” (Primo and BU Libs website) to track use better (also Law, Theo and Med sites?)
                                                                                                        Implementing goals (eg what subpages are branches wanting to push), setting goals given the BU libraries organization and heterogeneous users/uses may be a challenge
                                                                                                        Using existing dashboard templates for new dashboards
                                                                                                        Additional webanalytics tools to augment those analytics available from GA (eg clicktale.com, usertesting.com, qualaroo.com, compete.com)
                                                                                                        LP - Post-survey we should form a subgroup to look at implementing these suggestions

                                                                                                        DAG is looking to do some usability work specific to the BULS searchbox
                                                                                                        Subgroup for Faculty Survey review will meet this Wed 10/29

                                                                                                        UG survey:
                                                                                                        Intro needs provost/IRB review but otherwise done
                                                                                                        Minor wording and ordering changes on Page 1
                                                                                                        Reserves Question
                                                                                                        Reviewed 41a and 41b, two approaches to this question
                                                                                                         a - percentage use (tho may not add to 100)  
                                                                                                         b - last use snapshot 

                                                                                                        Broadly, we preferred 41a. We established that the scale we want is use-frequency, though there are concerns about how students will interpret that.

                                                                                                        Existing subquestions 1 and 2 (print and online reserves) can be combined, but for “find library resources on your own”  separate into "from library" vs "purchase"
                                                                                                        1. Reserves 
                                                                                                        2. Blackboard 
                                                                                                        3. go find your own readings (from library)
                                                                                                        4. purchase

                                                                                                        Wording note: Must make textbooks inclusion/exclusion unambiguous

                                                                                                        May not want this exact Q on grad survey b/c of multiple roles grad students may have (as both teachers and students), and because law/med/theo's approaches to collection/reserves may differ from Mugar & branches' (eg Law buys textbooks)

                                                                                                        **PILOT this question early and revise as needed

                                                                                                        Individual libraries at BU section
                                                                                                        Law name change – CHECK ALL SURVEYS FOR THIS (RS/ER)
                                                                                                        Skip logic magic was demonstrated
                                                                                                        Changes: seating/desks, dropped security, added outlets

                                                                                                        Whirlwind update from Grad Survey review group


                                                                                                        Assessment Meeting Minutes 10/6/2014

                                                                                                        Present: Linda (head), Brendan (minutes), David, Ellen, Steve, Konstantin, Sarah
                                                                                                        Next Meetings:
                                                                                                        • Undergraduate Subcommittee: Friday, October 17th, 12:30-2:00 @ SEL 
                                                                                                        • Graduate Subcommittee: Thursday, October 16th, 2:00-3:30 @ Law Conference Room 
                                                                                                        • Assessment Committee: ???

                                                                                                        Action Items:
                                                                                                        1. Find things to improve about the graduate/faculty surveys.
                                                                                                        2. Review undergraduate survey post-edits [[ ignore bracketed sections ]]
                                                                                                        3. ???

                                                                                                        Review of Changes to Undergraduate Survey 
                                                                                                        • Linda agrees to work on the introductions for the surveys. Shooting for ”5 Ipad minis for undergrads, 4 for grads, and 3 for faculty.”
                                                                                                        • Put display logic on a single “Individual Libraries” page so it pops up in page and makes the survey feel more condensed. 
                                                                                                          • POST-MEETING UPDATE: This isn’t possible due to display logic glitching. We’d have to change our format to make it work.
                                                                                                        • We should double check the surveys for the naming conventions for the libraries. With Law being renovated, should we change its name in the survey? It seems like we’ve been using the colloquial names rather than the official names in some places.
                                                                                                        • Section headers were added and embiggened.
                                                                                                        • Included pop-up question for people who have used a bibmanager, asking which ones specifically.
                                                                                                        • Added Steve’s reserve question. Undergrad subcommittee will iron out the wording/options/likert format.
                                                                                                        • Do we want a question about mobile devices? 
                                                                                                        • Question: Will the Qualtrics mailer be blocked as spam? 
                                                                                                        Graduate Survey Collective Critique 
                                                                                                        • Check whether distance ed. Is capturable from the grad. info. The rest of the demographics can be removed.
                                                                                                        • “Are we asking open access questions for graduate students too?”
                                                                                                        • Remove primary discipline since we get that information.
                                                                                                        • Brendan and David agree to help the other subcommittees for consistency between surveys.
                                                                                                        • Consider separating BU Library Search from other databases (see Q39).
                                                                                                        • Maybe we should separate out service/facilities questions.
                                                                                                        • Remodel the favorite library questions after the Undergraduate Survey. Sarah: “Study Carrels” 
                                                                                                        Publicity Update – Sarah

                                                                                                        • Meeting with Mary McGowan’s Anita Greene Scholars. We can get both vertical and horizontal designs from them. We’ll try to get them on the bus too. Posters should come up first, and then emails will go out. And they’ll have them ready by Thanksgiving and the students will be available for rebranding through intersession. Design style is “like a comic book, but not like that dark stuff”. 

                                                                                                        General Notes and Miscellany 
                                                                                                        • Brendan and David agree to help the other subcommittees for consistency between surveys. 
                                                                                                        • We should consider looking for a new 9th committee member to replace Lisa.
                                                                                                        • Konstantin: “Should we start gathering reference use data for grads/faculty in the surveys?” 

                                                                                                        Assessment Meeting Minutes 9/29/14

                                                                                                        Present: Brendan, Dan, Steve, Lisa, Ellen, Russel, Linda (chair), David, Sarah, Konstantin (minutes), Ken Liss
                                                                                                        Next meeting: Monday 10/6, 2pm-3:30pm, Mugar Administrative Conference Room

                                                                                                        Action Items:

                                                                                                                 1. Minutes, announcements, changes to the agenda?

                                                                                                                  2.   Publicity update (Sarah)

                                                                                                                 3.   Full committee high-level review of the Graduate Student Library Survey

                                                                                                                 4.    Discussion about Undergraduate Student Library Survey (Undergrad sub-group)

                                                                                                                 5.    Next steps?

                                                                                                        • Timeline
                                                                                                          • We should submit to the IRB in mid-January
                                                                                                          • Before we submit we should receive approval from the Provost office for ALL files (i.e. publicity, conflict of interest, surveys, etc.)
                                                                                                        • Weekly meetings
                                                                                                          • We will be meeting weekly, not biweekly
                                                                                                          • The following groups will be responsible for reports
                                                                                                            • Faculty survey: Steve, Russ, David
                                                                                                            • Grad survey: Sarah, Konstantin, Ellen
                                                                                                            • Undergrad survey: Dan, Brendan, Linda
                                                                                                        • Report from the Undergrad survey group
                                                                                                          • Brendan shared the file "Suggested changes for the Undegrad Survey"
                                                                                                          • All agreed that we should review Intro, exclude undergrads >18 years old by email distribution, reorder sections (Resources, Services, Individual libraries)
                                                                                                        • All agreed that we should follow up with focus groups

                                                                                                        Assessment Meeting Minutes 09/15/2014

                                                                                                        Present: Brendan, Dan, Steve, Lisa, Ellen, Russell, Linda (chair), David (minutes), Sarah
                                                                                                        Next Meeting: Monday, 9/29, 2pm-3:30pm, Mugar Administrative Conference Room 
                                                                                                        Following Meetings: For rest of year, will be meeting every other Monday at 2pm, except for one Friday: 9/29, 10/17 (Friday), 10/27, 11/10, 11/24, 12/8, 12/22

                                                                                                        Action Items:
                                                                                                        Sarah: Remind reference librarians about the ongoing MINES survey
                                                                                                        Linda: Ask for input from Ken Liss on questions about instruction and services (Q8 & Q9 in undergrad survey).
                                                                                                        Steve: Propose question to capture undergraduate use of "reserve" materials
                                                                                                        David: Talk to JD about question on bibliographic managers, and which ones to list in question.
                                                                                                        Brendan, Dan, and Lisa: Work on undergrad survey for next meeting, including, comparing to MIT and UW surveys, looking at qualitative data from last survey, and proposing any needed wording changes.
                                                                                                        Sarah, Ellen & ???: Similar work (for future meeting) for grad survey.


                                                                                                        Dan announced improvements to the websites Google Analytics, which now track searches from across site, including branches.

                                                                                                        Russell introduced our newest member, Ellen Richardson, Assistant Librarian for Administration at the law library.

                                                                                                        MINES Update

                                                                                                        Brendan showed the current state of the survey. We have had almost 3,500 results so far, with an extremely high completion rate. Brendan has downloaded the data (so far) into Excel to do some preliminary analysis, including parsing the URL to let us see what resource is being visited. Extremely impressive.

                                                                                                        Brendan will share data with anyone on committee who is interested. We can look at it and use it, but remember it is preliminary and hasn't yet undergone complete analysis.

                                                                                                        Next Survey

                                                                                                        Linda reported that until we hear from Provost (nothing yet), we still don’t know for certain whether we will do everyone in next survey, or only faculty. We will finish work on updating undergrad survey; then do the faculty survey update, in case that is the only survey we administer.

                                                                                                        Decided that for each survey we will do pass as a group to make sure we have right questions, without wordsmithing. Then break into groups to look at wording and compare to other surveys.

                                                                                                        Undergraduate Survey

                                                                                                        Sarah suggested we may want to speak to Ken Liss to find out if the questions on instruction and services (Q8 & Q9) are really the metrics he wants to track. Linda will do it.

                                                                                                        For the questions about facilities (Q13), we need to rethink our categories. "Furniture" can be confusing. Seating vs. Chairs. Outlets. Lighting.

                                                                                                        On Q18 regarding course reserves, Steve brought up wanting to know about where students are getting resources we traditionally think of as e-reserves (e.g., the library's e-reserves system, Blackboard, faculty web site, etc.). There was a lively discussion on whether undergrads (as opposed to faculty) are the right ones to ask, and whether a question can be written that captures what we want to know from undergrads. Steve will work on possible wording for question.

                                                                                                        Sarah remembered there being useful feedback on survey in qualitative data. When we break into groups, groups should look at qualitative data for their survey for comments about the survey.

                                                                                                        Q21: The question was changed from Yes/No to a scale; need to change wording of question to reflect change.

                                                                                                        Decided that if undergrads answer "Yes" to Q22 about using bibliographic manager, there will be a further question asking which one(s) they use. David will talk to JD about what should be on list.

                                                                                                        Because the wording for the comments boxes refers to the page titles, we need to add header to pages, or change wording of comment question.

                                                                                                        Some time was spent on the perennial use vs. importance vs. satisfaction problem, with no solution.

                                                                                                        Brendan, Dan, and Lisa volunteered to work on detailed wording for undergrad survey by next meeting. This will include comparing to MIT and UW surveys, and looking at qualitative data from last survey

                                                                                                        Sarah & Ellen volunteered to look at wording for Grad survey (for future meeting).

                                                                                                        Brief discussion of a replacement for the irreplaceable Lisa, who is leaving us. We are a large group, so we don't have to replace her. Linda suggests we consider needed skills in making decision.


                                                                                                        Assessment Meeting Minutes for 07/31/14

                                                                                                        Present: David (temp chair), Dan (minutes), Steve, Brendan, Lisa
                                                                                                        Next meeting: TBD - Doodle; Administrative Conference Room Mugar

                                                                                                        Action Items:.
                                                                                                        David will send a Doodle out to help schedule our next meeting due to August vacations.
                                                                                                        Brendan will continue editing the undergraduate survey draft on Qualtrics, according to decisions made today.

                                                                                                        Dan will share the archival surveys on Qualtrics, but set permissions so that no one can edit them.

                                                                                                        All can look at the undergrad draft to identify issues, better wording, missing questions/options/themes etc.

                                                                                                        - no announcements

                                                                                                        Discussion of 2015 Library User Survey Question Development
                                                                                                        Lisa lead discussion of the last sheet in this Google Doc, called "Comparing Under Q BU, UW, MIT". 
                                                                                                         - not much different from our to UW
                                                                                                        's survey: their survey is nice and very short
                                                                                                         - Answer rows on MIT survey Q2 are nicely specific.
                                                                                                         - We also looked at MIT Q11's layout (aware | importance).  We considered how this might apply for us, considering we often ask about use/importance/satisfaction.  We looked at our questions 9, 25, and 26 in comparison.
                                                                                                         - We have something very similar to MIT's Q12.
                                                                                                         - MIT Q16 + Q18 include library instruction outside of a class; probably not that useful for BU (could mean HGARC events etc?).
                                                                                                         - MIT did not ask overall importance of libraries
                                                                                                         - MIT Q24, the $100 to spend question.  Poorly worded and makes respondent add - not good for fatigue.
                                                                                                         - Lisa also noted that there may be some Qs from MIT not represented at all on the spreadsheet. She did not include them because there were no comparable Qs on our survey.

                                                                                                        Discussion of editable
                                                                                                        surveys on Quatrics
                                                                                                        David began this with the question of when, exactly, might it be favorable to combine the data coming from undergrads, grads, and fac.  There seem to be very few questions where we would want to do this, and without weighting the numbers for response rate, the numbers would be skewed.  We decided to begin composing Qs on three separate surveys.
                                                                                                        As we will begin editing the Qualtrics surveys, Dan will share the archival surveys on Qualtrics, but set permissions so that no one can edit them.
                                                                                                        We began looking at the faculty survey questions on Qualtrics.  We considered the frequency of physical and virtual visits, which were combined in 2010 but later separated, and we added a "during the current academic year" phrase as well.  We looked at the undergrad survey in comparison, and realized we should edit that first as it was the latest, and would need the least editing.
                                                                                                        Do we need to ask about majors again?  Used those to send comments to specific librarians, etc.
                                                                                                        Brendan noted the prevalence of the N/A option on many questions, and we noted the consultants advised having this "way out" for respondents.
                                                                                                        Our question 13 - piped text from previous question to set up to work in Qualtrics.  Steve is satisfied with the options on Q19, to look at changes over time.
                                                                                                        MIT asks about USE of more specific sources or collections than we do. We considered setting up our Q differently, but we do have many other statistical sources on collection usage, so we decided NOT to add any about specific USE other than the options we already have listed in the survey.
                                                                                                        Brendan suggested that we could create a numbering system that would indicate which questions are longitudinally useful.
                                                                                                        We completed an initial run-through of the questions on the undergrad survey, with some edits left for Brendan to finish up later, as he was "driving" the Pardee room computer and editing on the fly.

                                                                                                        Scheduling next meeting
                                                                                                        We looked at scheduling, but could not get around vacation schedules.  David will send a Doodle out to help schedule it.

                                                                                                        Assessment Meeting Minutes for 7/14/2014

                                                                                                        Present: Linda (chair), Brendan, Konstantin (minutes). Lisa, Russell, Steve, David

                                                                                                        Next meeting: Thursday, 7/31/14 at 12pm (at Pardee)

                                                                                                        Action Items:

                                                                                                        - Ask Bob Hudson to talk to Provost (Linda)

                                                                                                        - Look at the questions (David)

                                                                                                        - Discuss Brendan's action plan (all)

                                                                                                        - Take a look at Konstantin's analysis (all)

                                                                                                        - Update numbers on the spreadsheet (Dan)

                                                                                                        - Create the Google doc (David)

                                                                                                        - Wording, style, etc. - comparison of MIT and U of W - tabs: faculty (Konstantin), grad (Helen/Russ), undergrad (Lisa)


                                                                                                        Russ suggested adding Helen Richardson for membership on the Assessment Committee. (all agreed)

                                                                                                        Dan talked about Google Analytics - workshops - comparison functionality, bounce rate, exit rate.

                                                                                                        Dan reported on survey transfer. To access surveys on Qualtrics: go to bu.edu/qualtrics

                                                                                                        Discussion followed in regards to our survey plan

                                                                                                        Brendan's report: meeting with Pam Andrews (Lauren Hess replaced Pam in 7/14). Creation of panels. We discussed the advantages and disadvantages of working with the Provost office.

                                                                                                        Konstantin raised a question if BMC affiliates will be included in the panel.

                                                                                                        Assessment Meeting Minutes for 6/2/2014

                                                                                                        Present: Linda (chair), Brendan (helped with minutes), Konstantin, Lisa, Dan, Russell, Steve, Sarah, David (minutes)

                                                                                                        Next meeting: Thursday, 6/19/2014 at 2pm

                                                                                                        Action Items:

                                                                                                        • Dan: Transfer surveys
                                                                                                        • Lisa: Talk to Ken about Ithaka Q15.2 and 16.1 and Q4(?)
                                                                                                        • Konstatin: Looking at categories for each question. Talk to colleagues on medical campus about survey needs, and data management
                                                                                                        • Brendan: Working on Google Analytics
                                                                                                        • David: Help with display logic, ask JD and Jenna for input on data management question
                                                                                                        • Sarah: Ithaka Q 22.2: Sarah will compare to our faculty Q35 to see if there is anything we want from Ithaka. Look at reports to assess importance of disciplines
                                                                                                        • Russ: Examples of where WA survey wording better than ours

                                                                                                        Brendan: MIT Survey:

                                                                                                        • Some questions are much more specifically oriented. We’ve been trying to stay general with our questions but perhaps now we should start considering using more targeted questions.
                                                                                                        • Linda: “We should start keeping track of the “follow-up questions” like that one.
                                                                                                        • MIT Q13 (where to start looking for a specific ebook) could be useful as a simple statistic/longitudinal stat. Ithaka has a similar question.
                                                                                                        • Q24 (“How would you allocate $100 to improve library system”) Brendan likes a lot. “It’s fun!” Pretty easy to analyze too. Problems: many people can’t add up to 100, and the options may not be open ended enough.
                                                                                                        • Q25’s second question (“Are there other library services you think MIT should be offering”) is good in Linda’s eyes. It focuses on what the users want, not what we think they want.

                                                                                                        Dan: Transferring Surveys

                                                                                                        • Skip Logic causing problems.
                                                                                                        • Display Logic ALSO causing problems.
                                                                                                        • Dan will continue to tinker with help of David/Brendan.

                                                                                                        Russ: Comparing UW grad & undergrad surveys

                                                                                                        • “Is it typical for us to combine graduate and professional students?”
                                                                                                        • Sarah brings up concerns about using qualtrics for this survey if we’re having so much difficulty with skip/display logic. (“Well we have SurveyMonkey for another year if we need it”).
                                                                                                        • We really need to focus on the wording.
                                                                                                        • Possible wording: “Have you used the library’s services for an internship or sponsorship?” , or something to that effect.
                                                                                                          • “We followed UW’s model.”
                                                                                                          • We should include Marlene if we’re going to continue to discuss wording for Law users.

                                                                                                        Lisa: Ithaka Survey

                                                                                                        • Ithaka Q 7.3: possibly useful for follow-up to survey
                                                                                                        • Q 13.4: Possible springboard for question regarding publishing Steve: compare faculty attitudes towards publishers vs. libraries for publishing advice, etc. Russ: should library be responsible for some of these issues?
                                                                                                        • Q 15.2: Question about whose responsibility it is to teach information literacy
                                                                                                        • Q 16.1: Bring Ken into discussion about whether we want something like 15.2 and 16.1
                                                                                                        • Q 22.2: Sarah will compare to our faculty Q35 to see if there is anything we want from Ithaka.
                                                                                                        • Q 6.5 and 8.3: David will ask Jenna and JD to look at data management questions and give advice

                                                                                                        Assessment Meeting Minutes for 05/19/14

                                                                                                        Present: Linda (chair), Brendan (minutes), Russell, Dan, Sarah, Konstantin, Steve, David, 
                                                                                                        Next meeting: Monday, June 2nd, 1:00-2:30; Administrative Conference Room Mugar

                                                                                                        Action Items:
                                                                                                         - Russ: Compare UW grad & undergraduate surveys.
                                                                                                         - David: Compare undergrad & grad BU reports.
                                                                                                         - Dan: Transfer all other BU surveys
                                                                                                         - Konstantin: Compare the other BU surveys to each other.
                                                                                                         - Brendan: Compare the other BU surveys to each other & troubleshoot Qualtrics issues.
                                                                                                         - Sarah: Determine any problems with ceasing to collect discipline information & survey keeper

                                                                                                         - Brendan received a promotion to Assessment Data Specialist (or something like that).  Now we can offload all our projects onto him!

                                                                                                        Plans for New Survey:
                                                                                                         Linda brings up two important questions:
                                                                                                        1. How will we go about keeping track of our ideas/things to update or edit?
                                                                                                          • If we use the minutes, we'll need to be really organized.
                                                                                                          • Survey Keepers (see Wiki's side bar) are used to keep notes about conventions we've agreed upon, past-encountered problems, and survey decisions to avoid in the future.
                                                                                                          • General consensus to use minutes.
                                                                                                        2. Should we do one per year, or a three-in-one survey every three years?
                                                                                                          • Steve: Our response rates may change if we don't actively target a single user group.
                                                                                                          • David: If we put out a single report at the end for all three, it won't be targeted enough.  We should put out individual reports for separate user groups and perhaps an executivesummary of all three.
                                                                                                          • Sarah: We can filter the survey by user group.
                                                                                                          • Longitudinal statistics casually discussed.  If we want to start recording longitudinal stats, we need to make the changes with this report.  They have been used by other libraries and yielded interesting results.
                                                                                                          • If we do one survey per year, we're putting out one report per year as well.  Worries about survey and analysis burnout float around.
                                                                                                          • David: We could also have the home page link separate surveys for other user groups (everyone likes this idea a lot)
                                                                                                          • We could try using display logic in place of skip logic to bypass the skip issues.
                                                                                                          • General agreement on 3 surveys in one year, each tailored and targeted toward the specific user group, linked from a main survey page, and individual reports instead of a comprehensive report.
                                                                                                        Dan's update - Uploading Old Survey to Qualtrics:
                                                                                                        • Differences between the two:
                                                                                                          • The introduction transferred over into Q1.  Not noticeable from user's screen.
                                                                                                          •  Problem where the respondent cannot use the back button over any logically skipped survey content.
                                                                                                        David's update - Reviewing Survey Reports:
                                                                                                        • The reports referenced every question, but there wasn't a lot of deep analysis.
                                                                                                        • "Part-time" faculty wasn't a particularly useful demographic.  We should exclude it next time.
                                                                                                        • Question: are adjuncts considered part-time?
                                                                                                        • Q3 wasn't particularly useful either (whether faculty taught or used federal aid)
                                                                                                        • Perhaps we can simplify by not collecting department information.  Sometimes departments are fuzzy with self reporting anyway.
                                                                                                        • Brendan: "Can we extract info from email?  MIT collected no demographics and used email addresses to create demographics."  IRB would have a conniption fit if we started collecting personal data like that.  The anonymity would not exist anymore.  Not worth the time.
                                                                                                        • We can use conditional questioning to work around the parts people find confusing (e.g. with Med & BMC/school related fields)
                                                                                                        • David: "I want to ask about data management in the open access section"
                                                                                                        • Ranking importance is fine, but what we really could use is frequency data. 
                                                                                                        • We should add "in the last year" and drop "academic year" in place of "last year" so the data is current and not related to past experiences.
                                                                                                        Russ - Analyzing UW Survey:
                                                                                                        • "UW's survey is about the survey taker.  Ours is about what we think is important to them."
                                                                                                        • We need to package questions from their vantage point.
                                                                                                        • UW's survey is clean and minuses.  Ours can be wordy at times.  It has its plusses & minuses.
                                                                                                        • They included phrasings like "Thinking back to your last..." kind of questions.  May be more useful for infrequent services.
                                                                                                        Out of time. Next meeting decided for June 2nd.

                                                                                                        Assessment Meeting Minutes for 04/03/14

                                                                                                        Present: Linda (chair), Dan (minutes), Russell, Brendan, Lisa, David, Konstantin
                                                                                                        Next meeting: Thursday, May 1st 1-2:30; Administrative Conference Room Mugar

                                                                                                        Action Items:.
                                                                                                        Linda will renew Survey Monkey.
                                                                                                        Brendan will create a google doc in case groups wish to post comparison info before meetings.

                                                                                                        Dan will transfer the prior BU faculty survey from Survey Monkey to Qualtrics (2 copies).

                                                                                                        All can begin looking at other survey instruments (UW, MIT, Ithaka) for helpful comparisons.

                                                                                                        - Dan announced that Feb 27th marks the 1st full day of external link tracking through Google Analytics (GA).  One can view stats on internal and external links for any page on our site, though those numbers exist in two separate places within GA.
                                                                                                        - Linda announced that she and Tom C., Steve, Stephanie, and possibly David and Brendan will be attending a session at the Hyatt Regency on Academic Analytics.  The session will specifically be aimed at librarians.

                                                                                                        OCLC Assessment event update
                                                                                                        Linda will serve on a panel at an OCLC meeting at Brandeis on April 22nd called "Getting the Right Fit: Tailoring Assessment Strategies for your Library."  More info is available here.

                                                                                                        MINES is proceeding very smoothly now.  We have had a 0% drop out rate.  We need to think about how to back up the data periodically.  David believes we can download the data at any time, and we agreed that perhaps a schedule should be followed to download and backup the data periodically, although IS&T seems to trust the cloud.

                                                                                                        Faculty Survey planning
                                                                                                        We began planning for a faculty survey, not in exact order of the agenda, but the info is here below.
                                                                                                        Game plan- general, but not unanimous, consensus to perform surveys individually instead of grouped. We also decided to renew Survey Monkey (Linda will do this) once more as the deadline is fast approaching and it may come in handy
                                                                                                        Developing the survey - will follow these steps
                                                                                                        comparing to other surveys - Small groups will do initial comparison work and then bring suggestions to an AC meeting for all to discuss.  Brendan will create a google doc in case groups wish to post something before meetings.
                                                                                                        a.  Dan will transfer the prior BU faculty survey from Survey Monkey to Qualtrics and look for any differences (particularly in skip logic).  Dan will create 2 copies on Qualtrics - one to edit and one for posterity.
                                                                                                        b. Linda and Konstantin will then look over comparisons from our own grad and undergrad surveys
                                                                                                        c. Next, Lisa and Brendan will look at comparisons to the MIT survey
                                                                                                        d. Lastly, Russ and possibly others will compare to UW and Ithaka instruments
                                                                                                        2. final pass thoughts - examine possibilities of skip logic, questions or other items to eliminate to keep the survey short, and look through our survey keepers list
                                                                                                        iii.     Timeline - We're thinking about Spring 2015.  Russ expressed that we should be mindful of the timing, as Law faculty will be relocated during Spring 2015. Survey fatigue
                                                                                                        IRB - Linda will accomplish
                                                                                                        v.     Publicity/Recruitment - poster from Anita Greene group and emails
                                                                                                        Launch - nothing noted
                                                                                                        Analysis, quantitative - nothing noted
                                                                                                        Analysis, qualitative - have no one currently for this. there is the possibility of going back to tag previous surveys.
                                                                                                        Report writing - will likely rely on in-house talent.
                                                                                                        Staff involvement in changes - we need to examine this more.


                                                                                                        Assessment Meeting Minutes for 01/29/14


                                                                                                        Present: Linda (chair), Brendan (minutes), Russell, Lisa, David, Steve, Konstantin, Sarah, Dan

                                                                                                        Next meeting: Friday, Feb. 14th (Valentine's Day) 2-3:30; Administrative Conference room Mugar

                                                                                                        Action Items:
                                                                                                        - Notify Tim that Pubmed (and other resources?) should be placed behind EZproxy
                                                                                                        - LINDA: Call Amy about list being proxied. (DONE 2/3/14- LP)


                                                                                                        - Russell will be joining us henceforth as a new member of the Assessment Committee. Welcome!

                                                                                                        Tuesday, 1/21 Phone Call w/ UMass Amherst’s Rachel Lewellen

                                                                                                        Present were David, Steve, and Sarah and Linda.  Overall, a very positive, informative call.  UMA started out with a year long implementation and collected data from all users who started Ezproxy sessions during two, 2 hr. periods per month. In UMA's second year-long implementation of MINES, they switched to n = 140, and then reduced finally to n = 120. It’s important not to change n too far along so that data won’t have to be excluded, although Terry Plum implied that collection data can be adjusted if a change to N is needed during an implementation. Over 70% of surveyed users completed the survey at UMass (!!!), no complaints registered, not even about frequency. UMA did not have a comment box, but did have an email link. They only received one email. UMA included a question about why survey taker is using the resource.  Shared graphs are promising; of note the URL collection stats of those surveyed generally reflects the URL collection stats of the entire user-base (ie statistically relevant data).  Very promising overall!

                                                                                                        Let’s apply this to our MINES! 

                                                                                                        - We can use the URL data that will be gathered, but it will involve more involved analysis than other sources of data.

                                                                                                        - Konstantin: “We should use ‘BMC Resident’ for primary status question”.  Everyone agrees with the tactful word choice.

                                                                                                        - Google Chrome is being problematic for everyone with scrolling issues (only for most recent update).  It may interact with survey if someone tries to get to bottom of primary status list. “Maybe we can change the number of visible items on the dropdown list” suggests David.  Brilliance!  Later, “Alas, we cannot do that” informs David.  Foiled by technology again.  We shall investigate during test period whether Chrome will be problematic

                                                                                                        - “High school student shouldn’t be at top of list. It’s not ordered by age (Brendan grumbles), so we should order it by the groups most likely to use it, ie put High school students below staff on the drop down list.  General consensus, Brendan dissents vehemently.   Too bad for him.

                                                                                                        - David’s suggestion to get the sponsored question to drop down works.  An excellent solution to the problem we were having.

                                                                                                        - “ask@bu.edu” to be changed to “assess-l@bu.edu” so assessment committee members can personally field survey questions.

                                                                                                        - On criteria for assigning survey-testing: “Perfect for those who have miffed you!”
                                                                                                        - Linda: “We can’t go live until we’re sure the Med campus and Pubmed work.  We need to get Pubmed behind EZproxy.  And guide writers will need to update PubMed links.  Also many Law resources won’t be EZproxied.

                                                                                                        - Thoughts on n:

                                                                                                           * Larger student base and greater usage means n can be larger (or smaller? No, larger… we think).

                                                                                                           * We can change n within the first month if our number doesn’t suffice.

                                                                                                           * Linda: “Let’s go around and see what everyone thinks is reasonable.” Crickets

                                                                                                           * After some deliberation, Brendan: “100 or whatever is reasonable”.

                                                                                                           * Number-crunching Steve: “100 gets us just enough data with 70% expectation rate.”


                                                                                                        MINES Testing

                                                                                                        Directions will be on the site.  Mugar to have two demo sessions on Friday and Monday, one in the morning, and one in the evening.  Everyone else to schedule demos for the other branches. Remember, we want to test on as many platforms as possible, even mobile ones.  Also we need to test for blank survey results.  Proposal: Go live on 11th @ 12:00? … eh, maybe not.  We’ll see.



                                                                                                        -       Steve, on n: “Yes means no, and no means yes”

                                                                                                        -       Russ, on difference between Law Library and other libraries: “We actually provide service” (excellent smack talk, if I may say so)


                                                                                                        NEXT MEETING?

                                                                                                        -       Linda: “How does Valentines day work for everyone?”  All concur. There will probably be Valentine’s goodies, and Brendan will probably bring something (he didn’t mention this though). 

                                                                                                        Meeting adjourned.

                                                                                                        Assessment Meeting Minutes for 01/16/14

                                                                                                        Present: Linda Plunket (chair), Dan Benedetti (minutes), Russell Sweet, Brendan DeRoo, Lisa Philpotts, David Fristrom, Tim Lewontin
                                                                                                        Next meeting: Wednesday, Jan. 29th 2-3:30; Administrative Conference room Mugar

                                                                                                        Action Items:.
                                                                                                        Linda will work out what staff should test which databases (refnews/alphabetical?)
                                                                                                        Linda will change wording on MINES survey as outlined in 'B' below
                                                                                                        Sarah will be writing up the instructions for staff testing
                                                                                                        Konstantin will answer this question: If asked to give their primary status, would faculty on the Med campus be confused between a 'faculty' option and an 'affiliate' option?
                                                                                                        David will look at Qualtrics closely to see if we can easily get the URLs patrons will be forwarded to on completion of the MINES survey.
                                                                                                        All who can should attend a call with Rachel Lewellen, Assessment Librarian from Umass Amherst, Tuesday (01.21, 10am)

                                                                                                        - Linda reported the library wide meeting was a good success.
                                                                                                        - We've been invited to an undergrad faculty council, and a presentation will be given by Tom C. and David.
                                                                                                        - Brendan was asked by Dan to help with the Google Analytics initiative.
                                                                                                        - Today's meeting will be about MINES, so read this data visualization article in preparation for the next meeting:
                                                                                                        Azzam, T., Evergreen, S., Germuth, A.A., & Kistler, S.J. (2013). Data visualization and evaluation. In T. Azzam & S. Evergreen (Eds.), Data Visualization, part 1. New Directions for Evaluation, 139, 7-32.

                                                                                                        MINES Initiative updates
                                                                                                        Nearly ready to implement, but need to set up staff testing on a test website.  We will be looking for any minor issues, probably with 'refnews' staff, with Linda working out which staff should test what databases.  Sarah will be writing up the instructions for staff testing, and all should review them.  Russ also notes we should start the surveying early enough before finals, or alternatively wait until early June.
                                                                                                        A) Some institutions are putting open access databases behind ezproxy in order to track through MINES.  David reminded us that many patrons go straight thru google to resources, so we won't be able to get them.  Tim pointed out that we don't send pubmed thru ezproxy because of the way their website used to work, but we can now.  Because of possible complications with redirects, we should carefully test the performance of very important databases like pubmed. 
                                                                                                        B) We worked on the wording of the MINES questions.  Terry Plum had two suggestions on wording: 1-that we don't need an 'other' on q2.CORRECTION BY LINDA: TERRY'S COMMENT WAS NOT THAT WE DIDN'T NEED IT BUT ASKED IF RESPONDENTS WOULD KNOW THAT OTHER REFERRED TO OTHER-AT-BU.  2- on the 'purpose' question, to include 'research unfunded (faculty)', because it is primarily faculty that do research and she wanted to distinguish this from scholarship.   We decided to change this to "unfunded research and scholarship" and not include '(faculty)' or '(students)' in answer options.  We also decided to add an answer option for 'library staff'.  Brendan caught an incorrect plural of "affiliates'.  Finally, a question for Konstantin came up: If asked to give their primary status, would faculty on the Med campus be confused between a 'faculty' option and an 'affiliate' option?
                                                                                                        C) Two places are trying to get answers on what resources patrons are using, thru MINES: Umass Amherst and Univ of Toronto.  There will be a call with Rachel Lewellen, Assessment Librarian from Umass Tuesday (01.21, 10am) that all who can should attend.  Umass is also increasing the number of times the survey will be filled out, by having the survey pop up more often.  We decided that we should also track resources, and David will look at Qualtrics closely to see if we can get this easily, as it seems should be possible.