E-Resource Usage Statistics Q&A

Black = Questions Asked; Blue = Answers to Questions; Green = Answers that changed after further updated from Ex Libris

Red = Questions still outstanding; Orange = Question deferred to "Ask an Expert"

General Situation:

Response (3/28/16) via Brandon: Ex Libris and ProQuest are currently reviewing their 3 statistical packages (UStat, Alma Analytics and Intota) and determining what they want to do with them in a merged Ex Libris/ProQuest environment.

Response (6/29/16) via Brandon: Ex Libris does not plan to offer a data loading service as a free or paid service to Alma customers for several reasons (mostly financial) and would go so far to opine that they would probably not offer it to current SS/PQ customers for much longer based on their response to this question at the Exec meeting.

Detailed response to some specific questions: PDF distributed by Patrick (6/29/16)

Migration/COUNTER Support Questions

Whether/how to migrate current and historical usage data stored in USTAT into ALMA Analytics

Response (3/28/16) via Brandon: ~~Any usage data or SUSHI configurations setup in the test instance will be overwritten during the final load in Spring 2017~~. Ex LIbris is currently planning to merge/bring UStat into ALMA, but has not yet addressed what that means for data currently hosted within UStat.

Follow-up (9/14/16): Any progress or clarification on whether current USTAT data (and/or configurations) can/will be automatically loaded into ALMA? Or best practices for migrating usage data prior to go live date? (Or can we assume we will need to download USTAT data ourselves and manually upload it into Alma.)

Response (9/23/16): You will manually define SUSHI accounts per vendor. When you go live, a weekly job will be scheduled to harvest the files via SUSHI. Only new files received after the SUSHI account is added to the vendor account in Alma and the scheduled job is running will be uploaded. Any older files you wish to make available in Alma will need to be uploaded manually.

Response (9/26): SUHSI configurations will be retained at cutover

Response (10/14): Campuses using USTAT will have their COUNTER data migrated between October 10 and 22; Campuses who did not use USTAT will only have data received after a new SUSHI account is added to the vendor account in Alma. Any non-USTAT older files will need to be uploaded manually.

Migrating/re-establishing SUSHI configurations (e.g. from current USTAT instances to ALMA)

~~Ex Libris Response (3/28/16): Any SUSHI configurations setup in the test instance will be overwritten during the final load in Spring 2017~~

Follow-up (9/14/16): Any progress on whether current SUSHI configurations (e.g. from USTAT) can/will be loaded into ALMA automatically, or will we need to re-enter each account manually?

Response (9/23/16): SUSHI accounts will need to be added manually, and defined per vendor. They will be retained at the cutover (10/14).

Availability of COUNTER usage statistics via Alma

Response (3/30/16) via Brandon: I will ask ExL to update us on their plans in as much detail as they can currently provide as well as address what functionality will be present in our test instances.

Response (9/23/16): CSU can access COUNTER report service on ALMA starting in October 2016 (Email on 10/4 indicated functionality was activated on that date; Email of 10/14 indicated de-activation to load data from USTAT for 10/14 to 10/22)

Response (ExL HQ email to Brandon, 10/14):

- Alma Analytics COUNTER functionality should be active starting October 23 (and COUNTER data will be retained at cutover)

USTAT to Alma Analytics Cutover for COUNTER Data - now set for October 23

Response (3/28/16) via Brandon: Ex Libris announced in Feb 2016 that the ALMA roadmap for 2016 Q3 includes uploading and SUSHI retrieval of COUNTER usage data moving from UStat to ALMA. As part of this transition, UStat will no longer be available to ALMA customers.

Follow-up (9/14/16): We are waiting to learn whether a potential cutover from USTAT to Alma in 2016 Q3 applies to CSU libraries since we will only have a test instance of ALMA at that time. (E.g., will CSU libraries continue to use USTAT instead of Alma Analytics for COUNTER Data through June 2017).

Response (9/23/16): The Calstate institutions should have access to COUNTER report management in Alma in October (initially the 4th, updated to the 23rd to accommodate USTAT data migration).

Follow-up (JB/ND 9/26/16): I'm still unsure if the COUNTER data loaded from October 2016 through May 2017 will be deleted or not during the final load. We've been told all SUSHI data would be wiped clean on final load, but we've also been told we can start using ALMA COUNTER in October 2016.

Response (9/26/16): SUSHI Configurations will be retained from testload to "Go Live," and COUNTER data ~~will not be retained~~. will be retained at cutover (ExL HQ email to Brandon, 10/14)

~~Follow-up (9/26/14): Since all COUNTER data loaded during testload (October 2016 and Summer 2017) will not be retained, will we continue have access to USTAT through summer 2017?~~

Follow-up (ExL HQ email to Brandon, 10/14):

COUNTER Data availability via Alma Analytics should be live on October 23
Existing COUNTER data will be migrated from USTAT (October 10-22)
COUNTER data on Alma Analytics will be retained at cutover

Migration of USTAT Data - COUNTER Data

Response (10/18): ALL COUNTER Data existing on USTAT (as of 10/10) should be migrated to Alma (by 10/23)

Migration of USTAT Data - SUSHI Accounts

Response (10/18): SUSHI account information from USTAT is not migrated. Please do not create SUSHI accounts in Alma until after the UStat data has been migrated.

Migration of USTAT Data - matching with vendor records in Alma

To which vendor record in Alma will SUSHI accounts migrated from USTAT be attached?

Response (10/18): Incoming USTAT data is not matched on a vendor record. Incoming data Ustat will be retrievable/reportable in Alma Analytics.

Will we need to create vendor records for publishers who supply us with usage data but otherwise we have no direct relationship with?

Response (10/18): There is no specific “vendor” from the COUNTER report so it does not matter at all whether or not it appears in Alma. If you look at the usage data subject area of Alma you will see that there is no “vendor”. There is a “platform” and “publisher” but no “vendor”. The COUNTER report can be uploaded as “associated” with any vendor, and this is irrelevant as far as how it is filtered and displayed in Alma Analytics.

Migration of USTAT Data - Cost Information

Response (10/18): Cost information added to UStat is not migrated.

ALMA Analytics support for COUNTER reports

Response (3/28/16): Ex Libris is contractually obligated to support all COUNTER reports by June 2017, and Brandon and David will get a development update from Ex Libris soon. Currently, UStat supports COUNTER reports JR1 (with manual or SUSHI upload) and DB1 (manual upload only.) Ex Libris announced in Feb 2016 that the ALMA roadmap for 2016 Q3 includes uploading and SUSHI retrieval of COUNTER usage data moving from UStat to ALMA.

Response (6/8/16) via email: “In the initial stages support for PR1 and JR1a will be added. At a later stage additional types will also be supported. This is in addition to the existing JR1, DB1, BR1 and BR2.”

Follow-up (9/14/16): Can we get a list of ALL the reports in the COUNTER Code of Practice, both mandatory and optional, with the development status for each (i.e. implemented, expected Q_, on the roadmap). Specifically would like to get a sense of when the very valuable JR5 reports will be included.

Response (9/23/16):

Already available in ALMA as of August 2, 2016: DB1, BR1, BR2, JR1, JR1a, PR1
Planned (Stage 1 - "coming months"): JR2, JR5, DB2, BR3, MR1,
Planned (Stage 2 - 2017): JR1 GOA, BR4, BR5

Details found in Acquisitions/COUNTER powerpoint (August 2, 2016), Slides 7 and 9

Alma Analytics COUNTER/SUSHI have a similar "missing data" indicator as USTAT?

Yes

Support for loading non-COUNTER compliant usage data

Response (6/28/16) via Patrick: The library loads the COUNTER report data.

Follow-up (9/14/16): Does this mean there will be no support for non-COUNTER data

Response (9/23/16): Forwarded to Ask an Expert (Webinar at 7 AM on October 10).

Response (Ask an Expert): No support for home-made data, sticking only with COUNTER standard

Data-Loading Service Possibilities

Any potential for data-loading service and functionality via Alma (similar to Intota)

Response (3/28/16) via Brandon: Asked Ex Libris about the possibility of a data loading service being part of the new ALMA Analytics platform, and is awaiting a response. He pointed out that Ex Libris and ProQuest may not yet have determined the feasibility of bringing this functionality into ALMA. He assumes that the current ALMA Analytics functionality for data loading strongly resembles UStat.

Response (6/28/16) via Patrick: The library loads the COUNTER report data, not (as the question infers) “someone (Ex Libris staff or the existing PQWS folks)”. It is way more than twice a year. The SUSHI harvesting is weekly and can also be done “on demand” if desired. The manual uploading is “whenever you want”

Response (6/29/16) via Brandon: Ex Libris does not plan to offer a data loading service as a free or paid service to Alma customers for several reasons (mostly financial) and would go so far to opine that they would probably not offer it to current SS/PQ customers for much longer based on their response to this question at the Exec meeting.

Possible ProQuest Intota subscriptions

Brandon (3/28/16): In response to a query about providing a subscription to Intota for all campuses, Brandon mentioned that there are limited funds at the Chancellor's Office at this time and this request would be competing with other needs put forth by the libraries. He also cautions about adopting a third-party system that isn't integrated with ALMA since such a configuration defeats the purpose of a unified system. That being said, he completely understands the desire for a data loading service, and hopes we can provide Ex Libris with some impetus towards bringing the best of Intota Assessment into ALMA.

Response (6/29/16) via Brandon: Ex Libris does not plan to offer a data loading service as a free or paid service to Alma customers for several reasons (mostly financial) and would go so far to opine that they would probably not offer it to current SS/PQ customers for much longer based on their response to this question at the Exec meeting.

Desired Data Loading Functionality

(e.g. e-resource usage data management more similar to Intota than USTAT)

Responses received 6/28/16 in email from Patrick (with PDF attached to this page):

Data Retrieval Functions

Data Retrieval Service - customer maintains the log-ins and someone (ExL staff or the existing PQWS folks) retrieves the COUNTER reports twice per year and loads them.

Response: The library loads the COUNTER report data, not (as the question infers) “someone (Ex Libris staff or the existing PQWS folks)”. It is way more than twice a year. The SUSHI harvesting is weekly and can also be done “on demand” if desired. The manual uploading is “whenever you want”

2. Usage of the same title available on different platforms.

Question: Intota provides this, and it would be very helpful for determining whether usage of a title comes from the holdings in a database such as Academic Complete (with an embargo period of 12 months) or from the subscription you maintain to get access to the 12 months of content that isn’t available due to the embargo.

Response: Alma analytics can also show one title and usage on several different platforms. Email included a screenshot of a report for title ‘ASEAN Economic Bulletin’ shows level of use on EBSCOHost, JSTOR, GOLD, and ProQuest.

3. SUSHI service - clear error reports and ability to correct and then re-harvest

Response: Yes, data can be uploaded and then deleted and then re-uploaded. Error report provided by system (JB: seems similar to USTAT error messages).

4. Duplication safeguards - if the same report is uploaded more than once, or duplicated by a SUSHI harvest, the stats are not duplicated within Analytics

Response: If a user tried to upload duplicate data they get a message: The following overlapping data was found. Note that this data will be deleted if you continue with the current download process. (JB: seems similar to USTAT messages).

5. Ability to download raw reports from Analytics for local use

Response: Yes this can be done and it can be done in a wide variety of formats.

Analytics - Sample library measures and KPIs using Alma Analytics.pptx

Analytics - Overview of Usage Data subject area.pptx

Comment: examples are not exactly what we wanted to know, but answers to #7 cover some of it

6. Ability to upload reports to Analytics

Response: Must be loaded to Alma, with a daily ETL from Alma to Analytics

7. Ability to view the harvested or uploaded reports within the Analytics interface for troubleshooting

Response: Yes, this is possible. You can also view the specific reports in the Alma interface and in the case of “trouble shooting” it is recommended to use the Alma interface. (screenshot in PDF)

8. Authority title matching to reduce duplicate listings and ensure statistics are properly associated with titles held on multiple platforms; ability to manually match titles where automated matching has been unsuccessful

Response: There is no ability to manually match titles. We have not encountered situations where this has been needed. (screenshot of matching criteria in PDF)

Comment from NDM: withhold judgement on title authority matching until I see how it actually works in Alma--what they have set up does appear more sophisticated than the current Intota method

9. Ensure that ISSNs are consistently loaded in with 8 characters and include leading zeros.

Comment from NDM: ExL didn't answer this. I believe the question might be clarified in terms of how ExL handles different ISSN and ISBN formats. e.g., "0000-0020" vs. "00000020" or "978-0-03-029305-4" vs. "9780030293054." First, will Alma recognize that these pairs are the same ISSN or ISBN? Second, can it handle the leading zeroes, especially in non-hyphenated formats? e.g. when presented with "00000020," will it be able to see it as "0000-0020" or will it instead parse it as "20" because of the non-standard presentation? It should, of course, be the responsibility of the platforms/publishers to produce COUNTER reports that actually adhere to the standard, but it would be good to know how much flexibility is being built into Alma to recognize and handle the most common errors.

Follow-up (9/14/16): In regards to COUNTER data, can Alma handle importing standard numbers (ISBNs, ISSNS) that are not in the “standard” format. (E.g. with or without hyphens and/or recognize leading zeros)?

For example, one publisher may list an ISSN as "0000-0020", another as "00000020" – and similarly for ISBNs, one may list it as "978-0-03-029305-4" and another. "9780030293054."

a. First, will Alma recognize that these pairs are the same ISSN or ISBN?

b. Second, can it handle the leading zeroes, especially in non-hyphenated formats? e.g. when presented with "00000020," will it be able to see it as "0000-0020" or will it instead parse it as "20" because of the non-standard presentation?

(While it should be the responsibility of the platforms/publishers to produce COUNTER reports that are consistent with the standard, but it would be good to know how much flexibility is being built into Alma to recognize and handle the most common errors.)

Response (9/23/16): See the Ask an Expert (Webinar at 7 AM on October 10).

Reporting Functions

Incorporate data from Books in Print, Resources for College Libraries, and Ulrich's
Response: On the roadmap
Support for All COUNTER reports (including/especially Open Access)

Intota assessment can include usage for Open Access journals, or on open access articles inside non-open access journals. What will ALMA provide?
Response: Alma can get usage for data which comes from COUNTER reports. Alma can also get usage from the Alma Link resolver.
Comment - NDM: Were we asking about JR1 GOA reports? If so, that would be the best way to clarify this question. Since Carol also wants to know when JR5 will be supported, I think what we really need is a list of all COUNTER reports and their planned implementation.

A. Follow-up (9/14/16): Can we get a list of ALL the reports in the COUNTER Code of Practice, both mandatory and optional, with the development status for each (i.e. implemented, expected Q_, on the roadmap).

Response (9/23/16):

Already available in ALMA as of August 2, 2016: DB1, BR1, BR2, JR1, JR1a, PR1
Planned (Stage 1 - "coming months"): JR2, JR5, DB2, BR3, MR1,
Planned (Stage 2 - 2017): JR1 GOA, BR4, BR5

Details found in Acquisitions/COUNTER powerpoint (August 2, 2016), Slides 7 and 9

B. Follow-up  (9/14/16): Do we know if JR1 GOA reports (open access materials) will be supported? If so, will open access (JR1 GOA) usage be automatically filtered from the standard JR1 data, or will they overlap?
   Response (9/23/16): Support for JR1 GOA is planned for Stage 2 (2017). See Slides 7 and 9 of the Acquisitions/COUNTER powerpoint.
Response (9/23/16): Will Alma have any tool to de-duplicate possible overlap of JR1 GOA/JR1? Forwarded to Ask an Expert (Webinar at 7 AM on October 10).
Overlap analysis: journals, ebooks and streaming as currently on the Intota development timeline. Including functionality to check against external collections and unowned packages
Response: Overlap analysis can be done already. It can be done on collections owned by the library (portfolios in multiple collections) and it can be done in Alma analytics.
In Q4 2016 it will be possible to check overlap with portfolios / collections in institution and in Community Zone.
It is also possible in Alma and in Alma analytics to create reports on “duplicate titles” (titles with inventory both physical and electronic).
Ability to include/exclude platform & access fees from cost per use reports
Response: The price is taken from the invoice lines of the POL and cannot be selectively removed.
See https://knowledge.exlibrisgroup.com/Alma/Product_Materials/Overview_Materials/Presentations_and_Documents
Analytics - Overview of calculation of the cost per use in the Cost Usage subject area with DB and JR1.pptx
Analytics - Overview of Usage Data subject area.pptx
Intota’s planned proportional pricing enhancements to provide more accurate title costs across a package

Response: The Alma method of determining cost per use is very exact and precise.
See https://knowledge.exlibrisgroup.com/Alma/Product_Materials/Overview_Materials/Presentations_and_Documents
Analytics - Overview of calculation of the cost per use in the Cost Usage subject area with DB and JR1.pptx

Comment- NDM: Two things are not addressed here: Carol's question re how Alma handles journals outside the package (is their usage lumped in? what about their subscription costs? can they be excluded?) and this question of proportional pricing. I'm not quite sure I understand the proportional pricing part myself, though I imagine it is to do with the way journal packages can include journals that cost $250 and journals that cost $25,000, and they get treated the same way in cost/use. We might want to clarify exactly what the desired functionality would be here. Regarding the exclusion of non-package titles, this can be done with a filter in Alma Analytics now, but I have no idea how their associated costs are handled (I can't test because we didn't migrate acq data).

Follow-up (9/14/16): Cost per use

It looks like we can filter out “non-package” journal titles in standard Alma Analytics reports now. Do these filters work with the cost-per-use reports (where could we filter out non-package titles from cost per use calculations)?
Does Alma have any mechanism for handling proportional pricing in package or cost per use analysis? (E.g. – a journal package may include a title that runs $250 and another that runs $1750 – any way to weight these types of proportional costs?)
RESPONSE (9/23/16): Submitted these Cost per use questions for the upcoming Ask the Expert Session on Managing COUNTER Usage Reports in Alma.
See also these three document (via https://knowledge.exlibrisgroup.com/Alma/Training/Extended_Training/Presentations_and_Documents)
1.       Acquisitions - COUNTER diagram showing before and after the move to Alma from UStat.pptx
2.       Acquisitions - COUNTER report harvesting and uploading in Alma.pptx
3.       Acquisitions - COUNTER report harvesting and uploading in Alma status update Aug 02 2016.ppt

Desired functions not currently in Intota and unknown with regard to Alma Analytics:

"Out-of-the-box" reports corresponding to all collections questions for the major surveys
1. Most of them are now available, but a few of the trickier ones like title count are not
  
  Response: Alma analytics has a very vast, well used and proven set of OOB reports.
  
  Comment - NDM: The OOB reports are good (and they are), but the specifically referenced report, Title Count, is not among them, and I don't see one posted by the Community, either. That said, we are capable of coming up with some of our own, to share among the CSU as well as other Alma customers.
Automated collection analyses to aid in decision making, including benchmarking. May be part of ExL’s KPI work.
Response: Overlap analysis (portfolios in collections) and duplicate titles (physical and electronic) we have. Benchmarking and KPIs are on the roadmap
Clearly identify BR2 (e-book usage) report "Section Type" (e.g. chapter, section, page, etc.)

Response: A report of display titles for BR2 with the “Book Success Section Requests” is available in the PDF

Comment - JB: This appears they cannot identify the specific type of section being reported, but lump them all into the generic "section requests" category.
Comment - NDM: Looks like Analytics treats all BR2 data as generic "section requests."
This answers the question of how Analytics treats the data, but not our request to handle different section types differently or at least have a column that identifies section type and allows for filtering.

Follow-up (9/14/16): The COUNTER BR 2 reports seem to simply lump all types of sections into one field. Does Alma currently (or are there any plans in the future) to identify different types of sections reported in the BR2 counter reports, e.g., with filtering?
Response (9/23/16): Submitted to the Ask an Expert (Webinar at 7 AM on October 10).
User-configurable adjuster for estimated average chapter length to allow better comparison between e-book platforms that report BR2 usage by page vs by chapter
1. This would allow for more realistic "top performers" or cost-per-use reports across the collection
2. As it currently stands, the usage stats from aggregators like PQ and EBSCO, as well as encyclopedias and similar reference titles, overwhelm and hide the usage from PDF-chapter-oriented platforms (JSTOR, MUSE, most publisher sites)
Response: This does not exist and is not currently on the roadmap
Development plans include creating a place to store, manage and use data from non-bibliographic services (study room usage, lib instruction sessions, etc.) in various analyses.
Response: It is not possible to upload data directly to Alma analytics. The data comes to Alma Analytics via Alma. In the case of the usage data: COUNTER reports are loaded to Alma, then from Alma there is a daily ETL to Alma analytics.

Comment NDM: what we would like to know is whether it might be possible to manage non-bibliographic metrics through Alma. This seems like a complex idea, so I think we should clarify it for ourselves before asking again. For example, are we thinking this is something we'd want to manage through Booking? Perhaps some of these metrics would be better managed elsewhere; though I can see advantages to having consistency across campuses, Alma may not be the answer to everything.

Ask the Expert: Managing COUNTER Usage Reports in Alma -

Recording of October 6 "Ask the Expert" Session: http://knowledge.exlibrisgroup.com/Alma/Training/Ask_the_Alma_Expert/COUNTER_Report_Harvesting_in_Alma