Meeting Minutes

ULMS Analytics & Assessment Functional Committee met Wednesday, June 8, 2022 11am-noon

Attendees

Christina Hennessey (Chair/CSUN), Jung Ah Lee (SJSU), Shonn Haren (CPP), Kevin Phillips (Vice-Chair/CSUF),

Absent: Lisa Roberts (CSUS), Brandon Dudley (CO), Patrick Flanigan (SDSU). Notes by Christina Hennessey

Agenda/Discussion

1. Review of what the committee accomplished this year (2021-2022)

Goals we originally set for 2021-2022

We had great accomplishments on cleaning up the wiki and getting the Analytics folders clean and reorganized. We did a review of GreenGlass vs. Ex Libris Analytics and the pros and cons for each. We continue to share recent free outside webinars and training opportunities on Analytics and Assessment on the wiki home page for all the CSU (and the world) to use. We kept up with CSU Analytics & Assessment questions in Slack, on the listserv, and through email. Also not listed on the goals: we created a survey to get information about the top issues for Analytics across the CSUs, followed up on those issues, and also to get interest in trainings & future webinars.

2. Plans going forward for the committee

Said goodbye to outgoing members Christina & Lisa; Kevin will be the new chair; Jung Ah & Shonn & Patrick are staying on, and the committee will have Natalya Serge (CSUSM) and Stacie Jensen (Sacramento State) on the group next year. The new group will elect a vice-chair and make annual goals for the group in August or September 2022.

3. Updates on the Top 3 CSU Analytics issues given to Ex Libris

There has been no activity on the top 3 Analytics issues given to Ex Libris since our last meeting and since our follow-up with those that gave us more detail on those issues. Our reported issues were:

Analytics for Alma & Primo VE configuration changes and who changed them, retaining the whole history of each field change; Reporting from Primo Analytics too limited; Reporting on NZ-managed e-resources too limited

Kevin & Christina will meet with Ex Libris eventually along with other members of the team sometime in the coming months.

4. Recent Analytics & Assessment webinars

Kevin shared his thoughts on the June 6, 2022 webinar on Overlap Analysis, and Christina shared some notes from a June 6, 2022 workshop on "Diving into Data using COUNTER & Tableau" from Northern Ohio Techinical Services Librarians. Please continue to share any outside webinars you find at regional conferences that we can post on the wiki.

5. Backing up Analytics reports 

There was a helpful recent discussion (May 2022) on ALMA-L about backing up Analytics reports and protecting Analytics reports from deletion and overwrite through permissions. The committee will make a wiki page with this information and recommendations.

6. Humboldt name change in CSU+ reports/dashboards

The Resource Sharing group reported that some of the reports & dashboards needed to be updated to reflect the new name for Humboldt State University (Cal Poly Humboldt) as the data was now incorrect for that institution. Christina demonstrated how to update the bins in the reports to fix this problem. She found some of the them and fixed them but there might be more.

Next meeting: The group is dismissed for the summer. New Chair Kevin Phillips will reach out to the new 2022-2023 team in July 2022 to determine new monthly meeting times.

**

There was no meeting for May 2022.

**

ULMS Analytics & Assessment Functional Committee met Wednesday, April 13, 2022 11am-noon

Attendees

Christina Hennessey (Chair/CSUN), Jung Ah Lee (SJSU), Shonn Haren (CPP), Kevin Phillips (Vice-Chair/CSUF), Lisa Roberts (CSUS), Brandon Dudley (CO)

Notes by Christina Hennessey

Agenda/Discussion

1. ELUNA conference session related to Analytics has been announced for May 16, 2022 (titled: Updates: DEI, Primo, and Analytics) and will be held 10am-1pm Pacific Time online and is free. More information to come from Ex Libris on registration.

The only session on Analytics at the whole conference is a report from the Analytics Community of Practice on their work in the last year from 11:20am-11:50am Pacific Time.

The sessions in ELUNA in May 2022 are still only Ex Libris presentations. The details for ELUNA Learns, which has been presentations from Ex Libris customers in the last two years, have not been announced yet but you can expect Analytics sessions at those.

The group continues to post Analytics & Assessment-related webinars on the Analytics wiki; there have not been many since the beginning of 2022.

2. The Analytics Community of Practice had a session with Ex Libris product managers on March 15, 2022 where they presented upcoming features and changes to Analytics in: Leganto, Primo, Esploro, and Rapido; Christina will distribute notes to the group.

3. The change for Primo Analytics being one day behind to being two days behind was moved from April 10, 2022 for US sites to April 24, 2022. Ex Libris has done a good job of communicating these changes on the Ex Libris status page, in release updates, and other communication.

4. This group should be ready to work closely with the Resource Sharing Functional Committee in 2022-2023 for new reports needed for Rapido Analytics.

5. There will be a call for new members for this group soon from the Chancellor’s Office, please recruit folks for the committee to coming years, especially if they are new to the CSUs or have never been on this committee. Nominations will be requested from library deans or you can share your interest with bdudley@calstate.edu.

6. The group could not find an available and agreeable date for a summer Analytics webinar/event for June 2022, unfortunately. As the committee will have a new make-up starting in July 2022, that group will plan something for later in the summer or Fall 2022 instead based on the results of the December 2021 survey.

7. The group worked on the top 3 Analytics issues to submit to Ex Libris in more detail from the customer satisfaction survey. Christina will be reaching out to some of the survey respondents for more details and will rewrite the issues, impact, and what a solution would look like by April 22, 2022.

Next meeting: Wednesday, June 8, 2022 11am-noon

**

ULMS Assessment & Analytics Functional Committee met Wednesday, March 2, 2022 10am-11am

Attendees

Christina Hennessey (Chair/CSUN), Jung Ah Lee (SJSU), Shonn Haren (CPP), Kevin Phillips (Vice-Chair/CSUF)

Excused: Lisa Roberts (CSUS), Brandon Dudley (CO), Patrick Flanigan (SDSU). Notes by Christina Hennessey

Agenda/Discussion

1. Update on Shared Folders area in Analytics

The problem with automatic Shared Folders deletion of Analytics reports, folders and dashboards is finally solved. The items I had marked for deletion have not returned for three weeks now.

If you accidentally create a report in the Shared Folders/Community/Reports/Institutions/CalState that you don’t want there or don’t need anymore, please contact a member of the committee to remove it. Removals happen automatically each Friday evening.

Our cleanup project of Shared Folders/Community/Reports/Institutions/CalState is complete.

2. ACRL/IPEDS data support

The committee has been busy helping CSU folks who have questions about the ACRL/IPEDS reports. Those reports are not due until March 31, 2022 but most folks needed to give statistics to administration a few months ago. Please reach out if you still need help with these reports in Shared Folders/Community/Reports/Institutions/CalState/ACRL IPEDS Official ULMS folder.

The NZ report CalState – 41. Digital/Electronic Databases had an error with the prompt reported by a few institutions. Jung Ah fixed this for everyone on March 2, 2022.

3. Wiki updates/changes

 The group has made a lot of great changes to the ULMS Assessment & Analytics wiki page this academic year and our work is completed.

4. Analytics webinars/workshops

Christina did a demonstration of the inventory breakdown report that was shown at the end of the Cataloging Analysis Best Practices, Ex Libris, February 22, 2022 webinar.

Upcoming: The New and Improved Alma Overlap Analysis Tool, Ex Libris, March 29, 2022 7am Pacific Time.

5. Overlap analysis discussion

The committee discussed the current overlap and collection analysis tool and looked at the differences that will happen with the tool in the March 2022 release (Sunday, March 6, 2022). The improvements look great from this 5-minute preview video: https://youtu.be/UHF6Vu6A2Yw

The new tool will allow us to compare IZ, CZ, and NZ holdings.

The committee will try out the new tool before the next meeting and report out to the CSU what we find out.

Also a reminder about the overlap analysis reports in the dashboard area:

/Shared Folders/Alma/Titles/Reports has many overlap reports of different types, or you can see them all in a dashboard at: /Shared Folders/Alma/Titles/Dashboards/Title level overlap analysis.

P = print

E = electronic

D = digital

6. Start working on summer Analytics webinar

 We discussed more plans for a summer Analytics event based on the results of the December 2021 Analytics survey. June 2022 will be best so Christina will send a Doodle to check availability and check with CO on CSU-wide events.

Next meeting: Wednesday, April 13, 2022 11am-noon

**

ULMS Assessment & Analytics Functional Committee met Wednesday, January 12, 2022 11am-noon

Attendees

Christina Hennessey (Chair/CSUN), Patrick Flanigan (SDSU), Jung Ah Lee (SJSU), Shonn Haren (CPP), Lisa Roberts (CSUS), Kevin Phillips (Vice-Chair/CSUF), Brandon Dudley (CO). Notes by Christina Hennessey

Agenda/Discussion

1. Ex Libris 2022-2023 roadmap review

The committee reviewed the 2022 & 2023 roadmap emailed to all of us by Ex Libris on 1/9/22 for Analytics-related changes. https://knowledge.exlibrisgroup.com/Alma/Product_Materials/010Roadmap/Alma_Roadmap_Highlights_-_January_-_2022/Analytics

Some of the screenshots here are familiar from those we saw from focus group meetings in 2021 – new Analytics menu, easier to share with non-Alma folks. Customized dashboards are still in there despite the feedback from Analytics COP that the generic dashboards don’t really help customers as they too hard to modify for your local needs.

Some 2023 road map information here, I like this idea of ‘Information regarding patron facing letters and notices will be added to Alma Analytics’ so you can find something like ‘Number of Overdue Notifications per item counted by library’.

Electronic Resources Management also has some Analytics-related items

https://knowledge.exlibrisgroup.com/Alma/Product_Materials/010Roadmap/Alma_Roadmap_Highlights_-_January_-_2022/Electronic_Resource_Management

The Overlap Analysis Tool will be improved to include additional overlap comparing options and summary reporting, the Overlap Analysis Tool will be expanded to include print titles.

No other parts of the roadmap had Analytics or Assessment features.

2. Update on Analytics survey results

The team designed a two-part survey was out there for CSU responses from December 1 – 15, 2021, which covered both the questions we needed answered for the Ex Libris Analytics Chairs survey, and the questions we had about a CSU-wide Analytics event for 2022. We promoted the survey on the listserv and Slack. Analytics isn’t used by too many people CSU-wide so were happy with 15 responses and 12 campuses responding.

a. What was sent to Brandon for Ex Libris from Chair’s survey

The committee was sent all responses and we reviewed them for common themes as well as ideas for training. We also reached out to individuals that reported issues that were already fixed, so the survey was very useful in that way as well.

Our top five issues reported were:

1. Reporting on NZ-managed e-resources too limited
2. Overlap analysis - IZ, NZ & cost-per-use reporting still confusing or not available
3. Primo Analytics data is not reliable or blank or wrong
4. Analytics for configuration changes and who changed them, retaining the whole history of each field change
5. Have a full history of the users that updated an item/field, not just the last modified by

b.results for the Analytics event(s) for 2022

The committee reviewed all the results on time of year, format, interest areas, etc. The time of year of most interest was Summer 2022 so we will plan an online event for that time. There was most interest in these subjects: Data Visualization, Primo Analytics, and Electronic Resources Management, and format preference was both short presentations and breakout rooms on particular areas so we will have both. Stay tuned for calls for presenters as we plan this event throughout the spring.

3. Update on Analytics folder cleanup

Ex Libris is making progress on our issue of not being able to delete any folders or reports from the Shared Folders/Community/Reports/Institutions/CalState area. The process only runs on Fridays and some of the deleted reports & folders tend to reappear in the next Friday update. Once all the development issues are resolved here, the committee can return to our cleanup work of these areas. We reached out to the CSU Analytics community after the meeting to let them know of this ongoing problem, and to let Christina Hennessey know if you accidentally create a report in this area so she can get it deleted (maybe).

4. Any outstanding questions about ACRL/IPEDS data support or instructions?

The question of what to include in 60B was resolved. Patrick suggested we remind the CSU community that if they are running ACRL reports right now, there are some that needed to be run on July 1, 2021 to be correct, and the committee can help them with modifications to get this data (we did this after the meeting and a few sites did reach out for help).

5. GreenGlass discussion

One of the goals for the committee this year was to look at the OCLC GreenGlass product and how it supplements or complements Alma Analytics, and what is going on with various trials of GreenGlass throughout the CSU. GreenGlass is mainly a print analysis product but has some elements of overlap analysis with your e-resources as well. Christina has access to a GreenGlass account this semester and shared what it does with the rest of the group, and others who have been trialing the product shared experiences as well. Christina will have more training on GreenGlass later in January and will bring some questions and concerns from the committee to that training. Please share any concerns and questions you have about this product with our committee!

Next meeting: Wednesday, February 9, 2022 11am-noon (was moved to March 2, 2022)

**

November 10, 2021 11am-noon

Attendees

Christina Hennessey (Chair/CSUN), Brandon Dudley (CO), Patrick Flanigan (SDSU), Jung Ah Lee (SJSU), Shonn Haren (CPP), Lisa Roberts (CSUS), Kevin Phillips (Vice-Chair/CSUF). Notes by Christina Hennessey

1. Brandon Dudley from the Chancellor’s Office shared an update from a meeting he had on November 10, 2021 with Asaf Kline of Ex Libris on consortial needs. They will be looking for consortia to beta test some new “Out of the Box” Analytics reports for consortia on overlap analysis and the CSUs will help test. More to come on timing, the nature, and the location of these reports.
2. More discussion plans on a ULMS-wide session on Analytics. January 2022 is too soon for the ideas we have so it will be later in Spring 2022, we will send a survey to the ULMS shortly for ideas on format & topics.
3. The group shared their updates on the committee wiki, we are continuing to remove outdated pages, reorganize and update screenshots and instructions on useful pages. Many pages continue to focus on the 2017 migration (there is another place on the wiki to archive these pages, which we are doing) and ULMS-specific training (of which Ex Libris and user groups are providing updated training all the time). We continue to improve the wiki, not just for our own CSU folks, but for other Ex Libris libraries that discover our pages.
4. We continue to review and update the shared Analytics reports and dashboards in this folder and its subfolders: Shared Folders/Community/Reports/Institutions/CalState. There is a tedious process for renaming and deleting folders in the Shared Folders area that we are working through. We have reached out to former chairs with help on the history of this folder.
5. Review of some recent Analytics webinars: the use of the Primo Analytics example dashboard; and the use of drill-down reports. Links to the latest Analytics webinars can be always found on the committee wiki under sections “Upcoming Analytics & Assessment webinars” and “Recent Analytics & Assessment webinars”

Next meeting: Wednesday, December 8, 2021 11am-noon

October 18, 2021 11am-noon

Attendees

Christina Hennessey (Chair/CSUN), Patrick Flanigan (SDSU), Jung Ah Lee (SJSU), Shonn Haren (CPP), Lisa Roberts (CSUS), Kevin Phillips (Vice-Chair/CSUF). Excused: Brandon Dudley (CO). Notes by Christina Hennessey

Agenda/Discussion

1. Presentation by Christina Hennessey about changes Ex Libris is considering to the Analytics presentation inside of Alma (from a October 12, 2021 presentation to the ELUNA/IGeLU Analytics Community of Practice), got feedback from committee to take back to the October Analytics Community of Practice meeting
2. Other updates from Analytics Community of Practice (COP) such as the Inconsistent Facets report. The Events subject area is no longer going away December 2021 (it was going to be succeeded by the new “Systems Events” area, which was missing a lot that Events had) and the project is being revised by Ex Libris in conjunction with the Analytics COP.
3. Review of ELUNA Learns on Analytics from October 13
4. Continuing plans and discussion to update committee wiki, Analytics shared folders, and ACRL reports. We may have a CSU Analytics drop-in lab day in January 2022, where folks can bring their Analytics questions and the committee will work on them live.

Next meeting: Wednesday, November 10, 2021, 11am-noon

Sept 8, 2021 11am-noon

Attendees

Christina Hennessey (Chair/CSUN), Patrick Flanigan (SDSU), Jung Ah Lee (SJSU), Shonn Haren (CPP), Brandon Dudley (CO). Excused: Lisa Roberts (CSUS), Kevin Phillips (Vice-Chair/CSUF). Notes by Christina Hennessey

Agenda/Discussion

1. Introduction of Shonn to committee
2. Goals for the 2021-2022 committee finalized for presenting at Sept 15, 2021 ULMS Steering Committee Meeting

3. Reviewed committee wiki & documentation and discussed plan for updating

4. Reviewing ACRL Statistics Analytics folders and discussed plan for updating

5. Reviewing Analytics folders in general and discussed plan for updating and setting policies

6. Updates from ELUNA Analytics Community of Practice: Report of Inconsistent Measures/Facets and report on new System Events area

Next meeting: Wednesday, October 13, 2021, 11am-noon

**

There was no August 2021 meeting

**

July 14, 2021 11am-noon

Attendees

Christina Hennessey (Chair/CSUN), Lisa Roberts (CSUS), Patrick Flanigan (SDSU), Jung Ah Lee (SJSU), Kevin Phillips (Vice-Chair/CSUF), Brandon Dudley (CO). Notes by Christina Hennessey

Agenda/Discussion

1. Introductions of all for the new committee make-up for 2021-2022
2. Selection of Kevin Phillips as Vice-Chair for 2021-2022
3. Expectations of committee members and support for CSU Assessment & Analytics
4. Analytics in current Alma enhancement voting - there was only one: Reportable POL notes (7421)
5. Discuss ACRL stats – where are the shared reports, current status, reminding CSU folks to do stats as close to end/beginning of fiscal year as possible. Committee will review current concerns with shared ACRL stats Analytics
6. Update from Jung Ah on Silverchair/COUNTER5 issue
7. Ideas for goals for 2021-2022. We will firm these up at the next meeting (ran out of time)

Next meeting: Wednesday, Sept 8, 2021, 11am-noon

**

April 14, 2021

Attendees: Heather Cribbs, Christina Hennessey, Jeremias Vall, Lisa Roberts, Patrick Flanigan, Laura Krier, Brandon Dudley

1. SUSHI Survey: Heather put together a report based on the survey responses. Didn't get full participation this round, though we did the first time. Almost all campuses are using SUSHI to some extent. Difficulty of using Analytics is one of the biggest barriers.
   1. Recommendations: we could do a recorded presentation on cost-per-use (though Ex Libris offers these as well). Maybe want to offer an open forum where people could come and discuss challenges and solutions? Should schedule around the ELUNA sessions.
      1. Session on cost per use
      2. Session on SUSHI set ups for difficult vendors
      3. Possible to put together documentation for setting up SUSHI, at least for ECC vendors
2. Dashboard development:
   1. What information do deans actually ask for? Might be best to ask the people in libraries what deans ask for, rather than asking deans what they want?
   2. Can we find a way to make it easy for people on the campuses to provide that information?
   3. Perhaps it would be more useful to ask what the formerly-known-as-EAR committee wants and needs?
   4. ACTION: Reach out to the Resource Sharing group to find out what the usage is for their dashboard? Christina will do that
3. Overlap Analysis Joint Task Force:
   1. There hasn't yet been a meeting of the task force.
   2. We may need to postpone the work of the task force until the ERM committee is less consumed with CDI issues.
   3. ACTION: Laura will reach out to Kate (chair of ERM Committee) to see what her thoughts are.
4. Performance of Analytics post-migration
   1. Performance has improved, both in local and NZ analytics
5. Customer Services issues with Ex Libris
   1. NZ Analytics is improved
   2. Cost per Use is still confusing
   3. Overlap Analysis with the NZ is not possible
6. Committee Turnover:
   1. May might be our last meeting of this iteration of the committee, unless there are things to wrap up
   2. Christina will be the chair next year
   3. Call for new committee members will come soon

March 10, 2021

1. Reviewed thoughts on Analytics issues earlier today with Ex Libris and next steps were determined. Christina will re-do her reports with problems and send results. Laura and Heather will look at the power points from ExL and make changes/corrections based on those and update the Salesforce cases.
2. Laura has some feedback from the new usage data survey already but we'd like more data before putting together a report. A survey reminder will be sent. Heather will look at the results so far and summarize.

February 10, 2021

1. We need to complete the NZ Analytics performance testing before the Analytics meeting with Ex Libris, so they have the data for their analysis. The meeting is scheduled for March 10.
2. Report from the Data Viz task force was received. The recommendation was not to license Tableau, because it doesn't currently meet our needs. Creation of a data lake for data from all campuses was recommended. The report will be shared with COLD at a later meeting.
3. Heather and Laura will create a new usage data survey to be sent out before the end of February

January 13, 2021

1. Look at draft email re: ACRL reports for CSU librarians & staff and approve to send
2. Decided to put out another usage data survey, focusing specifically on COUNTER 5 usage.
3. Update on the Data Visualization task force:
   1. Waiting to get information from Lauren Magnuson
4. The committee should review the current reports in the ACRL folder to make sure they are correct and working
5. Commitee should continue to evaluate the new Alma Analytics platform after the July 2020 migration and the monthly updates since then

November 3, 2020

Guests: Brian Moore and Nikki DeMoville

1. ACRL Reports:
   1. Available for Group member field broke and there were continual problems with using the reports to get NZ data.
   2. There have been significant changes to what is being asked for by ACRL.
   3. Some of the campuses can't use specific fields to limit to campus because of overmatching. Institution codes are 100% distinct but group member codes are an issue.
   4. Questions about whether to include open access usage or not
   5. Some campuses don't have items attached to bibs, so title counts might be incorrect; volume counts might be wrong if barcodes aren't included
   6. Questions about data COLD might want that wont' be included in ACRL reporting
2. BD had a meeting with Ex Libris about pending development cases related to COUNTER 5 usage. They are considered enhancements and not bugs.
   1. Our satisfaction surveys are very important because they are taken seriously by Ex Libris. Brandon will try to capture information that he can out of salesforce cases and will verify that it includes all of the important issues.
3. Usage Data issues:
   1. R5 data numbers are a lot lower than last year.
   2. Newspaper usage is not included in journal reports, open access is not included.
   3. Lisa created some new test reports for R5 usage
4. Committee should evaluate the issues with usage stats and send a summary to COLD and a proposal about how we might deal with the problems. Should also document potential cross-campus comparison issues.

October 14, 2020

1. Data Visualization task force:
   
   1. Laura will reach out to Lauren Magnuson to ask her for the task force's report
2. Overlap Analysis:
   1. Laura will reach out to the ERM chair about forming a task force with members from both committees. Jeremias and Christine both interested.
   2. The outcome of the task force might be something Brandon could bring to Ex Libris to convince them of the need for this data in Analytics
3. Assessment of the new Analytics platform:
   1. We don't have empirical data re: the problems with the older platform. How are we going to evaluate the performance of the new platform? Error messages? User perception? Identifying reports that time out frequently?
4. ACRL reporting:
   1. Laura will ask Nikki to attend a meeting to talk about how the reports were constructed, decisions made, constraints and challenges.
   2. We should establish a group of folks to look at the existing reports and update them.
5. ECC Usage:
   1. Heather will email the chair of the new EAR committee
6. COUNTER 5 reporting tool
   1. Patrick will reach out to Tyler to see if he would be willing to present on this tool to the committee
7. eCAUG presentation on Analytics:
   1. How do you train the occasional Analytics user? What are the key things about Analytics that people need to be aware of?

July 1, 2020

• First meeting with the new 2020-22 appointed members, Jeremy, Patrick, and Lisa, with Laura Krier as chair.
• Alma Analytics migration discussion:
  • Analytics freeze scheduled for July 13, 2020. The migration itself is scheduled for July 22, 2020.
  • Drafted an email to send to all CSU with instructions for library staff and to ask for a person from each campus to let us know when the copying of Analytics reports is complete. The email was sent July 2.
• Primo Analytics migration scheduled for Q4 2020.

May 13, 2020

Attendees

Brian Moore (Unlicensed) (SDSU), Laura Krier (Unlicensed) (SSU), brandon dudley (Unlicensed) (CO), Heather Cribbs (CSUSB), Peggy Lau (CSUEB), Christina Hennessey (CSUN)

Agenda/Discussion

• Virtual Knowledge Days presentations are available; good information about changes to Analytics: https://knowledge.exlibrisgroup.com/Cross_Product/Conferences_and_Seminars/Knowledge_Days_-_ELUNA/2020_Knowledge_Days
• ERM Group wants to work on an overlap analysis between IZ and NZ content.
• Discussion about usage data and questions about whether to clean up old data
• Discussion about circulation data post-migration; most campuses saw drops in numbers from III systems to Alma systems.
• Next year's chair will be Laura Krier (Unlicensed) and vice chair will be Christina Hennessey
• Next year will probably be focused on preparing for migration to the new Analytics platform (new Oracle product, not just an upgrade of OBI), and verifying whether the migration resolves the issues we've had from the beginning.

February 12, 2020

Attendees

Brian Moore (SDSU), Laura Krier (SSU), Brandon Dudley (CO), Heather Cribbs (CSUSB), Peggy Lau (CSUEB), Christina Hennessey (CSUN), Tracy Gilmore (CSULB)

Agenda/Discussion

• Analytics performance: varying, no idea whether/what kind of load balancing they’re doing on the server. 
• Kudos to Heather on the Jan. 16th Tech Services Open Forum presentation on COUNTER 5 stats.
• ACRL Reports: 
• Using the Count Distinct formula to count titles seems to cause some weirdness
• Can’t figure out why adding this title count field causes collection countss to duplicate. [This was determined to be an issue with Oracle BIEE rendering results and exporting to MSExcel. Exporting to .csv showed correct counts.]
  
• Some questions about the filters for the ACRL merged report. Brian will continue to work on these to make them available for libraries. Creation date might need to be prompted, as well. Not sure if other campuses were having issues with the ACRL reports. 
• Data Visualization Task Force -- Laura will follow up with Lauren and/or Karen. 
• TS Open Forum -- someone should make an announcement again about the Analytics freeze at the end of February. Date is still not confirmed; Christina will find out what the new date is. Brian should be able to attend. 
• SUSHI: 
• Waiting for the February release to fix some of the SUSHI issues.
• Created a widget to show which vendors have COUNTER 5 stats
• Brandon met with Ex Libris to talk about the satisfaction survey
  • Delayed COUNTER5 support has been a drag on our processes as we've grown dependent on Alma to provide comprehensive usage reports. – 4
  • 
Scalability – the length of time it takes to run reports; large reports fail or take too long to run (particularly in the NZ) – 4
  • 
Reserve stats – the current set-up makes it very difficult to get results for items that do not circulate or were never used; correlating reading list usage with courses seems to be very difficult – 4
  • 
Primo – Zero result search analytics reports make no sense – when tried, all listed searches produce results (e.g.:  "bicycle" is listed in a recent report); I have learned from presentations & the listserv that the stats cannot be trusted – 4

• Main issue is the performance/capacity of analytics. 
• In the short term, Ex Libris has agreed to work with CSU to address issues identified in the fall survey. Issues that were identified: 
• Didn’t talk about how to introduce issues that weren’t mentioned in the survey, like documentation. 
• Documentation issues already have a process to be updated, by reporting pages that need work. 
• Could possibly also ask for a meeting at ELUNA with Audrey Ho to see if they want more feedback. 
• If the Primo issues can be scoped or defined a little less broadly, it would be easier to address. 

January 8, 2020

Attendees

Brian Moore (Unlicensed) (SDSU), brandon dudley (Unlicensed) (CO), Heather Cribbs (CSUSB), Christina Hennessey (CSUN), Tracy Gilmore (CSULB), Peggy Lau (CSUEB), Laura Krier (Unlicensed) (SSU)

Agenda/Discussion

Ex Libris Updates

• COUNTER 5 support implemented. Heather tested SUSHI connections and data loads; mostly worked for the vendors she tried. Ex Libris has created new canned reports for COUNTER 5 usage.
• There are new features in Analytics, including methods of handling OCLC numbers, reporting on non-transactional POLs in Funds Expenditures, new measures available in Fund Expenditures, and new fields and subfield data available in Bibliographic elements. Presentation is available online.
• Ex Libris will be rolling out updates to the OBI platform. Data visualization and OBI information is available online.

ULMS Wiki Updates

• Brandon updated and reorganized the wiki to follow a consistent information architecture.
• Please look over the Assessment and Analytics area to make sure no information is missing and links are working.

Analytics Issues

• Available for Group Member data in the NZ Analytics still not working.
• This information is available in IZ Analytics in the 'Details for Consortia' dimensions and does appear to be accurate. Users should continue to utilize the alternate ACRL reports provided for eResources until this issue is resolved.

COUNTER5 updates

• Heather spent a lot of time loading and testing COUNTER 5 data. Some vendors are not yet making R5 data available; some are no longer making R4 data available. Some vendors are making R5 data available but it doesn't seem to be compatible with Alma.
• Because R4 and R5 are so different, R5 data does NOT overwrite R4 data; both sets of usage stats are kept in Alma/Analytics. This raises questions about how total usage is calculated. Heather would like to do some comparisons to find out more.
• Heather will create a presentation showing R4 and R5 comparisons, and ask Luiz about presenting at tech services open forum. Heather will also look at creating a dashboard that will allow people to track SUSHI harvesting.

SUSHI Survey

• The survey mostly confirmed what we already thought.
• Brian shared the data with the ERM group.

December 9, 2019

Attendees

Brian Moore (Unlicensed) (SDSU), brandon dudley (Unlicensed) (CO), Heather Cribbs (CSUSB), Tracy Gilmore (CSULB), Christina Hennessey (CSUN), Laura Krier (Unlicensed) (SSU)

Agenda/Discussion

NZ Reports

• The Available for Group Member field in the NZ analytics is not working again. Brian opened a new case.
• This field seems to work in IZ analytics. Laura created some reports that seem to be accurate in IZ analytics.
• Once the bug is fixed, we should look at the differences between how this works in IZ vs. NZ and what the scope of the field seems to be.

SUSHI Survey

• Don't have results from all campuses, but we still have a lot of information.
• The majority of campuses are using SUSHI. Campuses are storing information in many different places, depending on the data and vendors.
• ERM task force is also interested in our data; we will share with them.

COUNTER5

• Heather had a question about SDLC usage stats. Some vendors don't provide individual logins for each campus, so we get the stats from the CO, but the data is inconsistently gathered and reported. Tracy has worked with the CO's office in the past on getting data. Heather and Tracy will work on creating a report of info to share with the CO about what we need and where the inconsistencies are.
• Heather also created a crosswalk of data between COUNTER 4 and COUNTER 5 and will share.
• We should post what we learn in the ERM workspace on the wiki instead of the Assessment and Analytics space, because people will be more likely to look there (but can link to it from the A&A workspace).

November 13, 2019

Attendees

Brian Moore (Unlicensed) (SDSU), brandon dudley (Unlicensed) (CO), Heather L. Cribbs (Unlicensed) (CSUSB), Tracy Gilmore (CSULB), Christina Hennessey (Unlicensed) (CSUN), Laura Krier (Unlicensed) (SSU)

Agenda/Discussion

SUSHI Survey

• Still waiting for results from three campuses, but good feedback so far.
• Brian and Tracy will start reviewing the data tomorrow and pull together a report to share outside of the Analytics working group, especially with the ERM group.

COUNTER 5

• Heather has been evaluating vendors to track who is using COUNTER 5. Very inconsistent use.
• Still need to set up a meeting with Heather, Christina, and Peggy.
• We won't be able to work with COUNTER 5 data in Alma until January, so there isn't really a rush.
• Updating SUSHI configurations to use pull COUNTER 5 and updating existing usage stats as needed will probably be a summer job.
• Most likely people won't have reliable data before ACRL reporting deadline (if ever).

ULMS Chair Survey

• Brian analyzed scores for the main issues people reported.
• Surprised by the number of comments related to reserves reporting.
• Cost per use data is inaccurate; Ex Libris is aware of the issues and working on them.
• Questions about the usefulness of Primo Analytics; people have found inaccurate data, especially related to reports on zero search results returned.

Barcode Query Tips

• Someone posted instructions on the Alma-L listserv about how to query Alma/Analytics using the barcode to pull information about alternate editions and multiple copies of items.
• Brian added this information to the Useful Code Snippets page on the wiki.

September 11, 2019

Attendees

Brian Moore (Unlicensed) (SDSU), brandon dudley (Unlicensed) (CO), Heather L. Cribbs (Unlicensed) (CSUSB), Tracy Gilmore (CSULB), Peggy Lau (Unlicensed) (CSUEB)

Regrets: Laura Krier (Unlicensed) (SSU), Christina Hennessey (Unlicensed) (CSUN)

The group discussed tasks associated with COUNTER5 and usage stats management.

1. SUSHI support - survey, access documentation
2. COUNTER5 - specs, comparison, ACRL integration
3. Usage stats - evaluation (what do I have? need?), application (what to do with it?), reports

Brian shared an impressive dashboard maintained by the Met Thomas J. Watson Library (III Sierra) with the group. Statistics are entered in a shared workbook and fed into Microsoft Power BI.

August 14, 2019

Attendees

Brian Moore (Unlicensed) (SDSU), Laura Krier (Unlicensed) (SSU), brandon dudley (Unlicensed) (CO), Heather L. Cribbs (Unlicensed) (CSUSB), Tracy Gilmore (CSULB), Peggy Lau (Unlicensed) (CSUEB)

Agenda/Discussion

Natasha Allen is leaving the SJSU for a job in Michigan. We're currently searching for a person to replace her on this committee.

Brian gave a brief overview of the informal committee meeting that took place at the CSU ULMS/IT/Scholarworks meeting in Northridge the week August 6-8.

We discussed a few projects that we may want to take on in this coming year:

• COUNTER 5 migration: COUNTER 5 support will not be available in Alma until January 2020. How will we help people work around this limitation between then and now, and support the use of COUNTER 5 when the functionality is available?
• COUNTER and Cost per Use in Alma Analytics: Ex Libris hosted a webinar describing some of the new functionality related to COUNTER platforms and cost per use. The webinar is available online at https://knowledge.exlibrisgroup.com/Alma/Training/Ask_the_Alma_Expert/9943_2019-08-13_-_The_COUNTER_Platform_and_Alma_Analytics_Cost_Per_Use_%E2%80%93_Recent_Developments.
  • We should considering bringing in someone from the ERM group and establishing a subgroup to look at this. We should develop a charge for a subgroup, contact someone from the ERM group (possibly Nikki?), view the COUNTER webinar, and look at data that we currently have in Alma.
  • Laura has done some work in Analytics recently assessing what usage data is currently available in Alma, and will share her process and reports. Wiki page link TK.
• Maintaining wiki documentation and making sure ACRL reports are up-to-date.

Ex Libris is planning to upgrade Oracle at some point. CO is closely monitoring and working with them to address some of our concerns and problems, especially related to working with consortial reports. Brandon would like the help of this group to surface current issues in as much depth as possible; we will talk more about this later in the fall.

Action Items:

1. Watch the COUNTER webinar from Ex Libris.
2. Laura will share her usage data assessment reports and process.

July 10, 2019

Attendees

Brian Moore (Unlicensed) (SDSU), Laura Krier (Unlicensed) (SSU), brandon dudley (Unlicensed) (CO), Heather L. Cribbs (Unlicensed) (CSUB), Natasha Allen (Unlicensed) (SJSU), Tracy Gilmore(CSULB), Peggy Lau (Unlicensed) (CSUEB)

Agenda/Discussion

Communication

• Slack is the primary method of communication.
• Also a mailto link to email the group on the ULMS Analytics & Reporting wikipage. 
• There is also a ulms-analytics mailing list.

Summer ULMS meeting

• There are several open labs scheduled; can committee members planning to attend the meeting host/participate? 

10:00 a.m. - 11:15 a.m.
11:30 a.m. - 12:45 p.m.
1:45 p.m.- 3:00 p.m.
3:15 p.m. - 4:30 p.m.

Fulfillment & Resource Sharing Lab
Group mtg.
Resource Management/Acquisitions
ERM

• Also a group meeting scheduled, but may not be needed if not enough people attend the meeting.

Current issues:

• The E-Inventory "Available for Group Members" dimension is unstable and repeatedly breaks (every odd-numbered month). Ex Libris is aware and has promised a fix this summer. Currently not working. 

Ongoing tasks: 

• Monthly archiving of the Calstate folder (/Shared Folders/Community/Reports/Institutions/CalState) within the Community folder in Analytics. Community folder is unstable; things frequently disappear or are edited. Brian creates an archive every month. 
• There was a meeting with Ex Libris at ELUNA where the issue of capacity in Network Zone analytics was raised. We're gathering use cases and instances of reports that don't run or that time out in the /shared/California State University Network/Uses Cases for Consortia Reports folder. 
• We've created toolkits for ACRL stats, as well as a Collection Development toolkit.

/Shared Folders/Community/Reports/Institutions/CalState/ACRL IPEDs Official ULMS folder/Run in the IZ

/Shared Folders/Community/Reports/Institutions/CalState/ACRL IPEDs Official ULMS folder/Run in the NZ

/Shared Folders/Community/Reports/Institutions/CalState/Assessment and Analytics/Collection Development Toolkit for the IZ

/Shared Folders/Community/Reports/Institutions/CalState/Assessment and Analytics/Collection Development Toolkit for the NZ

We may need to make changes to ACRL reports if ACRL makes changes to data they are requesting. Ongoing curation of these reports is part of our charge. 

Question about whether reports need to be run in July to capture volume counts at the appropriate time? At this point, yes they do. 

Should we send reminders to people to do this? 

Is it possible to modify the reports to an include a "creation date < July 1" filter? Might be possible; might be worth reviewing the reports. 

Best Practices document: This has been an ongoing project, and we'd like to continue to develop it. Includes things like privacy recommendations, retention policy recommendations, etc. 

Members should review this document before the next meeting to see if they have other ideas for additions, etc. 

There is a link for Useful code snippets and functions to use in Analytics on the wiki. This document lives on Brian Moore's google drive but is freely available via the ULMS link. Code snippets are added from various ELUNA presentations, with links to the presentation.

Steering Committee

• There is a new chair of the Steering Committee, John Wenzler. There is interest in starting to look more strategically at what we can do, rather than continuing to develop policies and procedures. What might this look like for our working group? 

June 12, 2019

Attendees

Nikki DeMoville (CPSLO), Brandon Dudley, (CO), Amanda Grombly (CSUB), Lauren Magnuson (CSUSM), Brian Moore (SDSU), Tracy Gilmore (CSULB), Peggy Lau (CSUEB), Karen Schlesser (SJSU)

Agenda/Discussion

Giant THANK YOU to outgoing members Amanda, Nikki, Karen & Lauren!

Policies status - Approved. Brandon will post to wiki

Available to Group Members data - It's back (for now although usually appears in even month releases)! Fix scheduled for this summers.

Reports for Ex Libris - 2 reports: Resource Sharing policy fulfillment (fulfillment), Resource Sharing replacement (fines & fees). Need a few more from other SAs by the end of them month. Document in Description, incl. contact info.

Task group updates

• Data Viz (Lauren, Karen) - COLD funded 1 yr. Tableau license. Experimenting to determine which subscription level is necessary. Working on Tableau toolkit and gathering data vis. experiences & best practices.
• Collection Development Toolkit (Amanda) - Posted - Instructions on wiki. Developed for IZ - starting point reports. Part 2: NZ required? Yes, for accreditation.

Summer meeting - Aug. 8th, We’ll have a room for labs with computers… There will also be an Assessment and Analytics group meeting time.

May 8, 2019

Attendees

Nikki DeMoville (CPSLO), Brandon Dudley, (CO), Amanda Grombly (CSUB),Brian Moore (SDSU), Peggy Lau (CSUEB)

Regrets: Karen Schlesser (SJSU)

Meeting with Yoel Kortick @ ELUNA

Brandon and David arranged for Nikki, Karen and Brian to discuss ongoing issues with Analytics. The group discussed:

• Performance - Many Alma reports are not usable in a large NZ such as Calstate or even at IZs with very large collections. They either take an inordinate amount of time or fail completely (even after verifying they work on smaller, filtered aggregations). Ex Libris has planned an upgrade of the Analytics architecture in order to improve capacity (obviously they are able to scale this to enable NA instance-wide reporting for their own purposes). Moving to Oracle BIEE 12 and considering alternate solutions such as Vertica. The complaint was also that Alma reports don’t appear to be tested at consortia scale before release. Action: Calstate will make a directory containing consortia report use cases. Ex Libris will take these reports and use them when testing the new improvements being done now.
• ‘Available for..’ - The lack of the ‘Available for Group Name / Members’ dimensions in the Titles SA makes accreditation reports rather complicated. These now only appear in the E-Inventory SA for electronic collections and portfolios.
• ‘Available for…’ availability - Nikki pointed out that this regularly disappears entirely in even numbered monthly releases. Yoel was genuinely surprised bye this. Nikki later shared the Case #’s.

Amanda noted that the shared print project with SCELC will make performance issues particularly critical.

Nikki led a review of policies covering NZ access, privacy, report sharing, backup, periodic auditing and report requests. These will be forwarded to the ULMS Steering Committee.

April 10, 2019

Attendees

Nikki DeMoville (CPSLO), Brandon Dudley, (CO), Amanda Grombly (CSUB), Lauren Magnuson (CSUSM), Brian Moore (SDSU), Tracy Gilmore (CSULB), Peggy Lau (CSUEB), Karen Schlesser (SJSU)

Agenda/Discussion

Salesforce case: Available for Group Members still unresolved.

TG Updates

Data Viz/SRU (Lauren) - Applied for a small amount of project money available from COLD for a Tableau license (pending).

Integrating Google authentication for SRU Lookup application to add account functionality. Solves the issue with reports taking a long time to run. They will be retained in the user account and an email sent when complete. Also, will include Lauren’s utility which sorts data from an uploaded spreadsheet by call no.

Sustainability/maintenance - Accounts will have to be approved and the application will need ongoing updates (e.g., Django, Python updates, SRU changes, etc.).

Collection Development (Amanda) - Ready to publish once "Available for" issue is fixed.

Enhancements

Although the functional groups were not assigned a formal role in the process currently the group made the following recommendations which will be suggested to CSU members: 6301, 6304, 5611, 5595

Policies, Best Practices, Procedures

Discussed policies. The group was encouraged to collaborate on the draft document.

March 13, 2019

Attendees

Brandon Dudley, (CO),Brian Moore (SDSU),Peggy Lau (CSUEB), Karen Schlesser (SJSU)

Agenda/Discussion

TG Updates

Dean's Dashboard/Data Viz update (Karen) - met with Roy Stripling (Chancellor's Office, Director, Student Success Dashboard Research Initiatives) to look at work he has done using Tableau (www.calstate.edu/dashboard). Deans conducting a survey of needs to inform the next iteration.

ACRL reports - January crunch time yielded a number of inquiries. The reports were invaluable tools.

SRU (Lauren) - Lauren and Jeremy Hobbs are working on an application. Expect a demo in the next couple of weeks.

Resource Sharing

• Dashboard: NZ/Shared Folders/California State University Network/Dashboards/CSU Resource Sharing

• Cost per Use study

• Frequently borrowed titles/subjects

• Looking at a variety of failed request/borrowing to create tools. e.g., immediate cancellations may point to config. issue

Per discussion on Alma-L: Title subject area in Analytics is updated on a monthly basis-- not daily (Noted in the Knowledgebase)

Group reviewed outline of Best Practices, Policies, Procedures. Brian is currently incorporating draft info. from the wiki, discussions, etc. Will open to comment soon...

February 13, 2019

Attendees

Nikki DeMoville (CPSLO), Brandon Dudley, (CO), Amanda Grombly (CSUB), Brian Moore (SDSU),Peggy Lau (CSUEB), Karen Schlesser (SJSU)

Agenda/Discussion

Request from RM group for training

• (from Luiz's email) Analytics Training for Resource Management - Committee discussed submitting requests for future training in Analytics in support of resource management statistics (e.g., How many titles and items added by location in a month; How many titles and items withdrawn by location; Introduction on the use of Library of Congress Classification (LCC) in Analytics: show how to better apply LCC field of LC Classification in dimensions).

Karen will share her reports for added/withdrawn, Lauren will cover LCC

Policies - All functional committees have been asked to review their old policies before they are moved into a new Policies space in the ULMS wiki

• Our group only has this one:  Working in the NZ Analytics instance

Nikki proposed updating the text and bringing it back to the Steering Committee for formal adoption

• Request from Brandon to clarify anyone can access NZ?, formalize process
  
• Privacy? User identity is anonymized in NZ but IZ acq payment data is not. Previous suggestion to require signed agreement not to expose data inappropriately deemed overkill. Can be included in policy & best practices.
  
• Brandon suggested policies/best practices are needed for: sharing reports, backup, cleanup, report requests - guidelines (Brian will work on)

Brandon: Analytics was a frequent topic of anxiety during recent campus checkins

• Nikki: Annual training? Project funding by CO has run out. Training by Connie Braun @ Ex Libris would be valuable.

EAR - their data interests lie outside Analytics currently. Continuing need for a liason - Tracy?

Community folders in Analytics

• Version control?
• Network backup will restore - weekly sync
• Strategies for making sure important reports are protected or backed-up
• Develop a policy, best practice and procedures
• Nikki will open Salesforce case to see how permissions can be edited (e.g., ExL reports)
• Karen: Analytics SU allows seeing when reports are used by whom

Policy on retention - 3 years?

• Idea Exchange: Include more identifying info with report data instead of user system #

Salesforce case: Available for Group Members still unresolved.

December 12, 2018

Attendees

Nikki DeMoville (CPSLO), Brandon Dudley, (CO), Amanda Grombly (CSUB), Lauren Magnuson (CSUSM), Brian Moore (SDSU), Tracy Gilmore (CSULB), Peggy Lau (CSUEB), Karen Schlesser (SJSU)

Agenda/Discussion

Updates on task groups:

Dean's Dashboard/Data Viz update (Lauren, Karen)

Dashboard shared with COLD at Winter meeting, Dec. 7th, 2018

• Rough comparison, Not recommended for ACRL sourcing
• Not all campuses report FTE in ACRL so headcount used throughout
• ‘# Presentations’ data needs refinement as campuses report differently. Refer to ref coordinators listerv.
• CSU+ Borrowing, Lending and Unfilled Requests compares to ACRL ILL stats from previous FY
• COLD was impressed and appeared to definitely support data collection standardization

Recommendations - Tableau lic. Req. + central resources

ACRL reports (Nikki)

Confusion over ‘official reports.’ Nikki cleaned up

/Shared Folders/Community Reports/Institutions/CalState/ACRL IPEDS Official ULMS folder

• /Run in IZ
• /Run in NZ

ACRL Toolkit on wiki

Benchmark Analytics (Nikki, Karen)

Activated in Alma (Benchmark analytics -- configuration)

Takes 30 days - Data previous to activation is aggregated, appears as “Unknown”

Comparison institutions are selected from same data center

Coll Dev toolkit (Amanda)

Collection Development Toolkit on wiki

/shared/Community/Reports/Institutions/CalState/Assessment and Analytics/Collection Development Toolkit for the IZ

Working on instructions

Not intended for subject specialists

Salesforce case updates (Nikki)

SUSHI harvesting

• new report types in Dec but JR2 missing from Analytics(?!)
• JR5 failing for multiple vendors

Group members broken AGAIN! - Fixed AGAIN

Amanda found suspicious # of owning members (+28?) in periodical count reports.

November 14, 2018

Attendees

Nikki DeMoville (CPSLO), Brandon Dudley, (CO), Amanda Grombly (CSUB), Lauren Magnuson (CSUSM), Brian Moore (SDSU), Tracy Gilmore (CSULB), Peggy Lau (CSUEB)

Regrets: Karen Schlesser (SJSU)

Agenda/Discussion

Updates on task groups:

Data viz (Lauren) - Working on dashboard demo. Have integrated data stores into single account so they can interact, incl. context for visualization elements.

• Suggestion to include major codes as an analyzed dimension in Tableau.
• Analyzing outlier data in order to notify sites. For now they will be excluded as ‘no data available.’

SRU (Lauren) - Webpage utility is 95% complete!

• Lack of ‘Available for’ data still an issue for SRU (fixed in Analytics) (Case #00607798 - Brandon will escalate)
• Q - What is being used as a matchpoint for older titles without ISSN, ISBN? A - Can use LCCN, OCLC#, ‘fuzzy’ title matching (+ format), etc.

Coll Development (Nikki, Amanda) - Developing base reports as tools for /shared/Community/Reports/Institutions/Calstate/Assessment and Analytics/Collection Dev folder. Sites can customize for local environment. Incl. instructions.

• Q - Will this overlap with the Acquisitions dashboard? A - Peggy will investigate whether the canned report is adequate or needs additional development.

Primo Analytics () - No report

Other? (Resource Sharing - Analytics TF is on brief hiatus)

Network Analytics - best way to continue training?

• Nikki presented a detailed NZ demo slide deck at the 2018 Southern California Technical Services Meeting.
• Tracy - might be helpful to do Ex Libris training as a group?

Benchmark Analytics - CPSLO is opted in, still awaiting data to populate (takes up to 30 days)

• Q - anyone else participating? A - not yet.

October 10, 2018

Attendees

• Nikki DeMoville (Unlicensed) (CPSLO), brandon dudley (Unlicensed), (CO), Tracy Gilmore (CSULB), Amanda G. (CSUB), Peggy Lau (Unlicensed) (CSUEB), Lauren Magnuson (Unlicensed) (CSUSM), Brian Moore (Unlicensed) (SDSU), Karen Schlesser (Unlicensed) (SJSU)

Discussion

SF case 00607357 - missing data from Available for Group Members

• Ex Libris has confirmed this issue
• Lauren notes the API / SRU is also affected - problem seems to be data coming from Alma

Fix for SUSHI Db reports was installed - not tested so far

ACRL/IPEDS: Ying has asked for page updates to the wiki - ACRL Survey Information

Collection Development Dashboard (Amanda, Karen)

• Still evaluating reports to include, to sched. meeting
• Solicit Slack input?
• COLD would like a December demo

Data Visualization (Lauren)

• Looking into Tableau licensing - it’s expensive
• Investigating public account
• Contacting data outliers
• Looking to incorporate FTE, ACRL stats
• Maintenance needs, grant possibility

SRU (Lauren)

• Using Django API, Google authentication to build web app using Python scripts
• Upload file, run report for overlap analysis, archive
• Also developing a Single ISBN lookup utility

Resource Sharing (Brian, Nikki)

• Functional reports are 1st priority (Turnaround times, Fulfillment rates)

Discovery

• No report

GreenGlass demo

• works out of the box
• compatible with SCELC
• not continuously updated (one-time load/analysis)
• Useful tool for identifying unique/rare material

September 12, 2018

Attendees

• Nikki DeMoville (Unlicensed) (CPSLO), brandon dudley (Unlicensed), (CO), Tracy Gilmore (CSULB), Amanda G. (CSUB), Peggy Lau (Unlicensed) (CSUEB), Lauren Magnuson (Unlicensed) (CSUSM), Brian Moore (Unlicensed) (SDSU), Karen Schlesser (Unlicensed) (SJSU)

Discussion

• Michael Germano was unable to serve. Welcome to Peggy Lau from CSU East Bay!

Benchmark Analytics

• Nikki will inquire on Slack regarding interest
• Brandon will check with other consortia
• Discuss with COLD in Dec.?

TF Updates

Resource Sharing (Nikki, Brian)

• Functional reports are 1st priority (Turnaround times, Fulfillment rates)
• Discussing other analyses

No reports from other groups - none have met since last month’s meeting in Long Beach

Data Visualization (Lauren)

• A few constructive comments were generated from the Summer mtg. debut
• Additional dimensions will be added to the wireframe
• Investigating Tableau license/quote
• Will look to incorporate Benchmark Analytics

SRU Searching (Lauren)

• Search/Retrieve via URL - returns XML (https://developers.exlibrisgroup.com/alma/integrations/SRU)
• Need to activate this Alma Integration
• Slow (1 search/sec), but works… No query # limit.

Use Cases

CSUSM

• ISBN Lookup - Bookstore provides spreadsheet of current textbooks. Lauren wrote a php script (Github: sru/isbn_lookup.php) to use ISBN to query Alma holdings. Working with Jeremy Hobbs to develop webform.
• OneButton - Combines multiple resource sharing options into a single button (https://github.com/PALNI/onebutton)

CPSLO

• Weeding list - Python script pulls in comprehensive data from IZ/NZ for bib, usage, NZ holdings (# of campuses) & eHoldings data to provide an overlap report comparing print to eHoldings

Interest in developing this as a utility. - Amanda will send data to Lauren and Jeremy for a report

SUSHI info

The spreadsheet of credentials was distributed to the TF. Very useful/accurate in general. Will be distributed to ERM contacts.

August 8, 2018

Attendees

• Nikki DeMoville (Unlicensed) Brian Moore (Unlicensed) Amanda G. brandon dudley (Unlicensed) Tracy Gilmore

Discussion

• Brief introductions, welcome new members
  
• Question posed to our group:

I am searching for best practices for streamlining usage statistics in Analytics, I am also interested in learning about people’s experience using analytics to pull COUNTER statistics."

"I am sending to you to address at the LB workshop and/or perhaps at a TS Open Forum zoom session.

How best to address this need

Instructions available for SUSHI

Need more detail

Nikki working on template for making sense of COUNTER reports

• COUNTER 5? (Tracy) - Ex Libris just catching up to COUNTER 4

Brandon - “Also need to watch for Alma support for COUNTER 5 - it’s “on the roadmap” but I’m sure won’t coincide exactly with the “release” of COUNTER 5”

Nikki proposed TS Open Forum in Dec(?) to remind everyone of setup

Gathering SUSHI login/credentials with Ying for spreadsheet publ.

• Benchmark statistics
  • https://knowledge.exlibrisgroup.com/Alma/Product_Documentation/010Alma_Online_Help_(English)/050Administration/050Configuring_General_Alma_Functions/Data_Sharing_Profile

• • Should all CSU campuses opt in to this?
    • Now that we have been live for a year…
    • Might facilitate cross CSU comparison
    • Is it available in NZ?
    • Consensus is this is a local decision but probably of interest to the Deans
    • Brandon shared a report on Benchmark Analytics for data cleanup
    • Nikki will enquire whether anyone else has turned this on
• Task forces
  • new member needed? (Alyssa has rotated off)
  • going beyond data gathering & visualization--possible to move into assessment (e.g. impact of instruction on use of OneSearch)?
  • Amanda mentioned they are moving away from OneSearch in favor of databases for undergrads. Nikki and Tracy say they are doing the opposite.
  • Instruction needs?
  • Mallory recruited 2 new members
  • Established naming convention, folder structure for vetting/development
  • Interested in KPI?
  • status report (non available)
• Discovery
• Resource Sharing (Nikki)
• Fulfillment (Brian)
• Data viz
• Collection Development Dashboard (Amanda)
• Requests for a finished product but little detail. Specific reports need to proceed a dashboard.
• New TF - Tracy, Amanda, Nikki
• Fullerton is presenting @ IGelU on Collection Reports… - Amanda will reach out.
• SRU searching (Nikki)
• can circumvent NZ Analytics limitations. Lauren has worked with this.

July 25th, 2018

Attendees

• Nikki DeMoville (Unlicensed) Karen Schlesser (Unlicensed) Brian Moore (Unlicensed) Amanda G. brandon dudley (Unlicensed) Lauren Magnuson (Unlicensed)

Discussion

Summer meeting (Nikki)

Poll of who will be attending the 1st day (Analytics Session) - all attendees will be there.

In the afternoon session, Megan will present 2:30 - 3:45 p.m. on Cost Usage reports (1st 1/2) and Fulfillment reports (2nd 1/2). Our drop-in session will be 2:30 - 5:00 p.m. We will need to coordinate attendance at Megan's session.

Dean's Dashboard (Lauren)

• Demo’d Tableau wireframe
• CSU-wide view but can filter by campus, time period
• Some infrastructure will be required for storing data and automation, may need system-wide Tableau lic.
• Recommend anonymizing and trimming data for the demo so that users focus on the tool rather than the (data) discrepancies.
• An LC breakdown is in the works
• Brandon will bring to COLD

Karen is working on a Collection Development dashboard for San Jose. Will put in shared folder.

Tech Services Open Forum

Luis has invited presenters for the fall

• Karen volunteered to show her Collection Development dashboard
• Others?

June 13, 2018

Attendees

• Nikki DeMoville (Unlicensed) Karen Schlesser (Unlicensed) Brian Moore (Unlicensed), Alyssa Loera (Unlicensed) Amanda G. Moon Kim (Unlicensed) brandon dudley (Unlicensed) Lauren Magnuson (Unlicensed)

Discussion:

• Group make-up will change, thanks to all who served and those continuing to serve
• Year-end-recap:
• Nikki provided an annual summary: https://docs.google.com/document/d/1MUfMaYQx0RTahjN_rWo3vu6CCpMMVJPMMYDQooS87gE/edit
• Action item for Committee: provide feedback by Friday, 6/15/2018 (the document is editable)

• ULMS summer meeting:
  • There will be an all-day analytics training on day 1
  • There will be hands-on help, especially with usage
• Usage stats help project:
  • Ying has contacted publishers and provided Nikki with logins/passwords for campuses

• Lauren Magnuson (Unlicensed) Action item: Lauren is working with the Acq/ERM group and others in CSU-wide Technical Services to get a list of contacts to distribute the user name/passwords (make sure to include Jessica)
• brandon dudley (Unlicensed) Action item: Brandon will investigate where the individual campuses’s ER administrative login/password info should be stored

• Alma vendor record bug:
• Currently cannot enter login information in the “Statistics Information” tab, only “Administrative Information” tab
  
• Fiscal close:
• Karen Schlesser (Unlicensed) and Amanda G. Action item: Karen and Amanda will work to gather stats in Alma where only a snapshot is available (e.g., portfolio counts for ebooks, ejournals, IZ and NZ) for the year-end close
• Task Force updates:
• Resource Sharing from Nikki: 
  • Nikki created a dashboard, still work in progress but available for logging in and viewing at the NZ level, demonstrating the impact on individual libraries
  • Lending seems more balanced (evidence of ROTA working perhaps)
  • Borrowing will probably require some assessment as to why some campuses have higher levels of requests
• Data Viz from Lauren:
  • Working on Dean’s dashboard (deadline: 7/15/2018)
  • Working on needs assessment and environmental scan (deadline: 6/30/2018)
  • Carve out some time during the analytics day to share prototypes during the ULMS summer meeting and provide opportunity for folks to comment during the open lab time,also announce the progress update during the announcements
• Primo Analytics from Alyssa:
  • Still looking at primo vs. google
  • Some mapping taking place but it’s documented on confluence
  • Will probably need to rope back to look at primo analytics and CSU+ data
  • Focus has been heavy on UX
• Fulfillment from Brian:
  • [Mic issues] Will provide an update next time

May 9, 2018

Attendees:

• Nikki DeMoville (Unlicensed) Amanda G. Brian Moore (Unlicensed) Lauren Magnuson (Unlicensed) Moon Kim (Unlicensed)

1.  Several campuses have no usage data at all in Analytics

• Discuss best approach to help them
• Some campuses are still setting up collections, POLs, and general Alma data and infrastructure, aren’t at the point yet where they can set up SUSHI or usage data infrastructure, may not know their account information or how to get started
• SDLC account information and setup for other institutions

• Nikki will ask Ying to contact SDLC package vendors for to ensure that accounts are setup for each institution and compile a list of usernames that institutions can use to reset their passwords (may need institutional emails for this?)

• Action item:  Craft an email with instructions for getting started and send to ERM group to distribute, gather pain points and get a sense of how we can help them; see how to identify campus primary contacts? See https://calstate.atlassian.net/wiki/spaces/URM/pages/151453722/CSU+Cataloging+Contacts for example (Lauren)

• Action item: Prep example report that institutions can use to see exactly what may have already been migrated for you or already exists in Alma (could also help with iterative setup) - dashboard that would show platforms and usage data in one view to look at data as it’s harvested
  • Create instructions for setting up a single vendor / manual upload and check results in Analytics (Link to/update existing Tips and Tricks)

• Recommendation that we standardize reporting codes, item types - campuses are using these in different ways
• This would help with ACRL reports and other systemwide comparisons/reports
• Should define POL types/standardized Alma fields for use systemwide
• Start with Acquisition data?  Develop example reports?
• Want to avoid forcing people to redo work

• Action item: Ask Acq/ERM group whether they would explore and recommend some standardized fields based on how people are currently using their reporting codes (Lauren)

2.  ELUNA

• SWIG meeting updates (Moon)
  • Working on evidence document to outline issues that Alma/Primo/Leganto users are experiencing
  • Ex Libris expressed commitment to moving issues forward; plans discussed:
    
    • No SQL access to live or copies of anyone’s data ever
    • No plans for analytics connected to sandboxes (v. expensive)
    • Supportive of institutions building their own data warehouses of Alma data
    • Random Analytics timeout - Ex L assured it’s been fixed; should be extended to 59 minutes - will not extend past an hour for security reasons
    • Added 9 new subject areas since January 2017 and will continue adding more
    • If you want new fields added put in enhancement request or work with SIWG to get them added
    • Certain fields are named differently between Analytics and Alma - Ex Libris is committed to fixing those discrepancies - submit support request
    • Cost Per Use is being completely revamped (!)
    • Many data elements will move to einventory subject area

• Useful things we learned and how best to share these (all)

3.  Summer training

• Developing plans (Brandon, whatever can be shared)

4.  Any other TF updates anyone wants to share, as time allows

April 11, 2018

Attendees

• Nikki DeMoville (Unlicensed) Amanda G. Brian Moore (Unlicensed) Karen Schlesser (Unlicensed) Lauren Magnuson (Unlicensed) Moon Kim (Unlicensed)

Discussion

Enhancements

Group reviewed the Analytics section in order to provide a select list to Brandon including:

1. Create a new field in Design Analytics for OCLC identifier 035 only (5619)
2. Enable more than 5 Local fields in Analytics(5637) is on the roadmap and should be disregarded included. (Moon subsequently spoke with the chair of the NERS Analytics Group. This was initially scheduled for H2 2017 on the 2017-2018 roadmap but looks like it’s been pushed back to H1 2018 on the 2018-2019 roadmap. Included so it will be a higher priority)
   
3. Nikki noted that Load usage data: Create different symbols in Missing Data page to differentiate between technical errors and no recorded usage(5618) appears under the Acquisitions section. Will be included.
4. Karen noted Provide Overlap Analysis Functionality for Print(5711) under Resource Management section. Will be included.
5. No NZ-level file sharing area in Alma Analytics for consortia(5611) (Moon submitted) - Currently NZ reports are only available in the NZ shared folder.
6. Timeout issues - Major impediment to new users particularly.
7. Add indicator for Course Reserve Loans(5595) - should be easier to report.
8. Retain Patron Statistical Identifiers(4986)

Dean’s Dashboard

Starting with ACRL - Karen

CSU+ stats are ready (need to run in NZ).

Lauren mentioned that there is currently a product for accessing CSU-wide ACRL data. However, these could use more context. (e.g., CO’s FTE #’s do not incl. Extended Studies/Open University)

Committee Reports

Fulfillment

Brian reported that the Fulfillment Analytics Group is brainstorming benchmark statistics which may be pertinent to the Dean's Dashboard. They are also developing Best Practices recommendations such as naming and folder organization conventions. Nikki has been maintaining a Tips & Tricks page on the wiki.

Moon will be attending ELUNA where the Analytics Group will be meeting with Ex Libris to discuss major issues such as timeouts and the failure of cost per use analyses.

Feb. 21, 2018

Attendees

• Nikki DeMoville (Unlicensed) Amanda G. Brian Moore (Unlicensed) Karen Schlesser (Unlicensed) brandon dudley (Unlicensed) Lauren Magnuson (Unlicensed) Alyssa Loera (Unlicensed)

Discussion

• ACRL stats
  • All turned in. Problems? Pomona requested assist for setting up Sushi harvesting. Others may as well (Karen will contact Luis)
• Discovery - Analytics Joint Task Force for Primo Analytics
  • Alyssa, Lauren - comparing Primo Analytics and Alma stats. Possible data analyst for CO?
• Resource Sharing - Analytics Joint Task Force
  • Brian, Nikki -
• Fulfillment - Analytics training
  • Amanda - Natasha & Matthew presented at Open Forum. 999 Restricted fields need to be defined as Local Param.
• Resource Management - Analytics training
  • NZ reports?
• Data Visualization
  • Lauren - 1st mtg. next week. (agenda) Possible grant for data analyst.
• EAR
  • Amanda - Collection dev reports?
  • EAR e-books project (Nikki) - Looking at JSTOR, Academic Press books (Proquest) title lists. Seek license/print availability among campuses. NZ Analytics doesn’t scale well with large data sets. DDA is not encoded anywhere. Different Withdrawn practices among campuses are problematic for NZ analytics. Lauren has been using the SRU integration for a similar project.

Policies

https://calstate.atlassian.net/wiki/spaces/ULMS/pages/108265513/Policies+Recommendations+and+Best+Practices

• NZ Analytics role
• Withdrawn items - impact on Analytics
• Other policies, Recommendations?
• Best practices for NZ use (versioning, confidentiality agreement)

Dec. 13, 2017

Attendees

• Nikki DeMoville (Unlicensed) Amanda G. Brian Moore (Unlicensed) Karen Schlesser (Unlicensed) brandon dudley (Unlicensed)
• Regrets:  Lauren Magnuson (Unlicensed), Alyssa Loera (Unlicensed)

Discussion

• ACRL: 
  • Webinar was well-attended; Ying Liu has been very good about copying us and adding our contact information to ACRL/IPEDS emails.
  • NZ Analytics suddenly has a large number of similarly-named ACRL reports, probably from other campuses trying to run reports; many of these will not give the desired results.
    • Action items:  Create an official folder with our recommended analyses (Amanda G.) and others in group will test to be sure results are consistent and work for other campuses
• Primo Analytics:
  • Neither TF member was present to report
• Resource Sharing TF:
  • Making progress toward rota balancing metrics.  Brian Moore (Unlicensed)has been working with Mallory DeBartolo (Unlicensed) to find an accurate metric for borrowing/lending stats (the basic Num Requests measure doesn't provide a distinct unduplicated count). Not possible in Analytics to create a report that uses more than one subject area and has filters that come from non-shared dimensions, so he is creating separate, appropriately-filtered reports for borrowing and lending and combining in Excel.
  • New request to create report to identify returned CSU+ books inappropriately reshelved by the borrowing library instead of being shipped back to the lending campus.  This is a common error that typically occurs if the patron has removed the book band or other ILL identifiers.  Nikki DeMoville (Unlicensed)will work with local ILL staff to test a report of titles that have been checked in but not received by the lending campus within a reasonable window of time and then share the results with the RS TF and hand off to Mallory for scheduling and distribution to other campuses as appropriate.
• Fulfillment:
  • Amanda G.is working with the group on training needs, has shared training resources pages from our wiki; Fulfillment group wants to do some analytics/reporting webinars
• Resource Management:
  • Agreed to do a TS Open Forum presentation on NZ Analytics after the first of the year.  Will call for volunteers when date is settled.  Everyone should be thinking about what reports would be useful to share with the TS audience.
• Data Visualization:
  • Will get going sometime early next year.  Discussed difficulties in getting access to Tableau on various campuses.  Problem may be license expense.  Per brandon dudley (Unlicensed) there is no consortial deal at this time.  May want to explore the idea once this group has some proof-of-concept demonstrations in place.
• Update from Brandon:
  • Deans interested in a "Deans' Dashboard."  Will need to gather information regarding what they would consider useful.
  • Deans interested in CSU+ metrics and assessment.  This is a high priority for the RS joint task force (rota balancing is highest due to operational impact).  Mallory has made good progress in identifying appropriate metrics.
  • Brandon will reach out to the COLD Assessment Team (CAT) to put them in touch with our group for Alma/Primo data needs.  Several members of CAT are from Cal Poly SLO, Fullerton, and San Jose, which may facilitate communication.

Nov. 16, 2017

Attendees

• Nikki DeMoville, Brian Moore, Karen Schlesser, Alyssa Loera, Brandon Dudley
• Regrets:  Lauren Magnuson, Moon Kim, Amanda Grombly

Discussion

• Amanda (via email):  working with Fulfillment group on Analytics training; looking at reports for ACRL
• Primo Analytics task group:  using Google Analytics to compare with Primo Analytics--want to understand better what Primo is reporting, and how trustworthy the data is
• Lauren and Alyssa have been looking at Tableau
• Discussion, suggestion that Data Visualization deserves its own task group
• Lauren, Alyssa, and Karen will organize, put out a call for additional volunteers outside the Analytics group
• Webinars:  TS Open Forum 11/16/17; ACRL Stats webinar 12/5/17
• Other task groups:  working on their own, no specific progress to report at this time
• Question raised by Amanda about training in tools like Excel
  • https://calstate.atlassian.net/wiki/spaces/ULMSR/pages/208338952/Training+Resources
• Discussion:  this kind of basic training is outside of our charge
• However, CSU does provide some resources like Skillport
• Group will develop a wiki page with training resources for various tools

Oct. 18, 2017

Attendees

• Nikki DeMoville, Brian Moore, Moon Joo Kim, Amanda Grombly, Karen Schlesser, Brandon Dudley, Lauren Magnuson, Alyssa Loera

Discussion

• Volunteers to work with:
  • Possibly doing some short demos during their weekly open forum
  • weeding?
• Fulfillment (Amanda)
• Tech Services (Karen, Brian)
• ACRL
• Status report
• Messaging
• What can we do in the NZ that will answer the same questions for all campuses, and how do we deliver that data?
• NZ analytics
  • Legacy, IZ - Physical Circ stats, Expenditures
  • Who needs?
  • When to use?
• getting some questions regarding when to use NZ or IZ
• probably need to clarify on the wiki
• Joint task force updates
• Discovery
• Resource Sharing

Sep. 20, 2017

Attendees

• Nikki DeMoville - E-Res & Res Sharing coordinator @ San Luis Obispo
• Alyssa Loera - Digital Serv & Technology, Head @ Pomona
• Brian Moore - ILS Mgr. @ San Diego
• Moon Joo Kim - Acquisitions Librarian @ Fullerton
• Amanda Grombly - Collection Dev & Mgmt coordinator @ Bakersfield
• Karen Schlesser - Library Data Mgmt Coordinator Tech Svcs @ San Jose
• Brandon Dudley - Project Dir., ULMS; CO committee rep
• Mike DeMars (guest) - Discovery Group Chair; Systems Librarian @ Fullerton
• Regrets:  Lauren Magnuson - Collection Mgmt & Tech Svcs, Head @ San Marcos

Introductions

• Briefly describe your background, experience, and interests, especially with regard to assessment and/or analytics

Meetings:  how often and when?

• Regular meeting? - Monthly
• Webinar outreach? - Open forums as needed
• Polls - Amanda will poll best times

Outward communication:  training sessions, open meetings, wiki, etc.

• Start with ACRL - identified as a training necessity in earlier assessment
• Wiki - Karen & Alyssa will help re-org
• Analytics listserv will persist
• Brandon will investigate hosting additional
• “Problem dropbox” for help with difficult reports/analyses
• Email alias of volunteer analytics experts (this group + others)
• Fed by Google form
• Email alias for this group

COLD Analytics needs

• Brief summary from Brandon
• What would a dean-level dashboard look like, and what might be the best presentation to work toward?
• anticipates the need for data driven decisions
• e.g., Fulfillment transactions, CSU+, etc.
• Integration with Tableau?
  • Note from Lauren:  FYI I am currently working with our Tableau admins to try and output data from Alma Analytics to Tableau, and we’re running into some issues sharing data from the Alma Analytics web connector to Tableau. I have a ticket in with Ex Libris about this - #00470713

Annual Stats task group

• Making sure CSU is ready for ACRL and other annual surveys
• Annual stats - Investigate running from the NZ
• ACRL - recommend a webinar (Karen, Moon, Amanda, Alyssa)
• Requires combining data from legacy sys. For this FY

Interactions with other groups

Analytics / Resource Sharing - Nikki, Brian

• Assess CSU+ system (e.g. rota balance, % requests fulfilled by CSU vs ILLiad, comparison to Link+)
• Assess CSU+ shared content (e.g. freq. borrowed items, common disciplines)
• Anya is already keeping some stats - cochair?

EAR

• How best to support EAR needs
• Supporting the collections Big Picture vs crunching usage stats
• Amanda is a member - EAR Analytics TF was discontinued in favor of this group
• Amanda will be this group’s liaison to EAR

Discovery Analytics Joint Task Force - Alyssa, Mike, Karen

• The Discovery Group would like to form a joint task force for Primo Analytics
• Mike DeMars (Discovery Group chair) presented the idea
• Group will review canned reports, needs assessment
• Contact Mike if interested
• Initial makeup of the group is Mike DeMars from Fullerton, Kenny Garcia from Monterey Bay, Ian Chan from San Marcos, Dave Walker from the CO, and Zach Williams from Dominguez Hills
• One or more volunteers from this group are requested (Alyssa, Karen volunteered)

Discussion of promoting the Ideas Exchange, coordinating NERS voting

• Brandon says a plan for consortial NERS voting is forthcoming; there might be discussions in the future to also coordinate NERS voting for individual campuses with ELUNA or IGeLU membership
• Alma users have 20 ‘tokens’ for ea. subscribed module (Alma, Primo, etc.) to vote in the Idea Exchange (and get them back if it’s implemented)